Reminder and Agenda for Confidential Storage Spec Call - Feb 25 2021
|
Dear all, This is a reminder that the DIF / CCG Secure Data Storage Working group weekly call will be happening on Thursday at 4pm Eastern / 1pm Pacific / 22:00 CEST. Meeting link: https://us02web.zoom.us/j/84828031746?pwd=V0xGTnJ2Zm15RHlSRFpNTlRPQzdLUT09 Specification: https://identity.foundation/confidential-storage/ Specification repository: https://github.com/decentralized-identity/confidential-storage Audio recordings and transcripts of previous meetings: https://dif.groups.io/g/sds-wg/wiki/19633 As always, the IPR policy requires that you can only make substantive contributions if you sign the IPR Release Form. Please follow the instructions at https://dif.groups.io/g/sds-wg/wiki/Home Confidential Storage Spec Call Agenda 1. IPR Reminder 3. Continuation of the ‘Division of Responsibilities between Hubs and EDVs’ discussion.2. Introductions and Re-Introductions PLEASE REVIEW: Daniel Buchner's proposed list https://hackmd.io/qClYLUPkQ7uf0r3_4O7BUQ 4. Issue review Thank you, The Chairs |
|
|
|
Manu Sporny
On 2/27/21 8:55 PM, Chris Were wrote:
I could break down the list we have discussed over the past few weeks andYes, this is a next step. Ideally, we'd map every feature in the spec today to CouchDB documentation. Chris, it would be good for you to volunteer to do this since you seem to be the one most driven to demonstrate that this is true. It's something we're going to have to document in time anyway, to demonstrate why the work needs to be done (or that W3C shouldn't waste their time on it because... CouchDB exists). Once you document it (not just the list of requirements we've been going over during the last month, but all the features in the existing spec as well), we can review it as a group to see if there is consensus. Zooming out a bit to look at the big picture... thinking about how SQL was standardized may help. SQL was first standardized by ANSI in 1986... but there were databases on the market for years that had the "relational database" feature set... relational databases existed in 1970, 16 years before the /first/ standard existed. SQL has been standardized ever since.. with the most recent version being SQL:2019. That's 35 *years* of standardization! Why are we still standardizing it!? To replay your point back to you, but in a different context: MariaDB (a popular open source relational database) covers many of the common use case needs for relational databases... so why do we need an SQL standard? With that in mind, some thoughts on the questions you asked: This process feels like attempting to reinvent the wheel and produce aIt always feels like that at first... like you're rubbing two sticks together to create a fire when we already have nuclear power plants. Why don't we just build a nuclear power plant? (Answer: there is no single specification on how you build one and some of the skills are so specialized that it's out of the reach of 99.99% of the population, which is not a good number if you're trying to document and teach people how to do something). The process of standards are not to innovate (at least, not primarily). It's to look at everything that's out there and try to standardize the simplest set of technologies that fit a Pareto distribution... that is, what 20% of features meet 80% of the use cases. The goal isn't to get the standard to support 100% of all use cases. I understand CouchDB is not a specification, but as an implementation it'sI have a vague concept of what your requirements are, Chris. :) I'm sure you don't have a solid concept of what Digital Bazaar's requirements are... or Transmute's... or SecureKey's... or Microsoft's... or Michael's. We're just scratching the surface, these discussions take a LOOOONG time to get to a basic understanding on everyone's *public* use cases. ... but the way we get there is to talk about it, and documentation of the sort you're talking about is vital to that process. Thoughts? -- manu -- Manu Sporny - https://www.linkedin.com/in/manusporny/ Founder/CEO - Digital Bazaar, Inc. blog: Veres One Decentralized Identifier Blockchain Launches https://tinyurl.com/veres-one-launches |
|
|
|
The lack of use-cases makes reaching consensus much harder since all we really have to build on is the charter. I spent some time today categorizing my personal data relationships and started a Use Cases document https://hackmd.io/q-uwOlisS0igaR2-vfXmlg Please edit or add to the categories from your perspective and also add use cases. I would like to discuss the authorization perspective in a call. If it is helpful to separate our work into EDVs and Hubs, then this thread https://lists.identity.foundation/g/sds-wg/message/82 also tries to address the authorization protocol(s). - Adrian On Sun, Feb 28, 2021 at 10:24 AM Manu Sporny <msporny@...> wrote: On 2/27/21 8:55 PM, Chris Were wrote: |
|
|
|
Chris
Hi All, The recent conversation discussing Hub / EDV spec has been extremely useful as it has helped shape the key areas the spec needs to consider and draw some clearer boundaries around requirements. That being said, it has reinforced for me how the majority of the items being discussed are already well implemented and battle tested within CouchDB (which already runs across multiple platforms and protocols). This process feels like attempting to reinvent the wheel and produce a sub-par outcome. I could break down the list we have discussed over the past few weeks and point to the relevant section of CouchDB documentation that addresses those requirements. I understand CouchDB is not a specification, but as an implementation it's pretty darn close to what we're looking for. It seems to make sense to have a meaningful conversation about how that gap could be bridged to speed up this process and result in a much richer feature set? I probably sound like a broken record as I've brought this up multiple times, but have not received any meaningful response as to why CouchDB, it's learnings and it's advanced feature set are being ignored. There's currently a tendency to put stuff out of scope because it's too hard / complex to work within the current implementations compliant with the spec. That feels a bit like the tail wagging the dog. Is that the real criteria we should be using? That's definitely a consideration, but more important is what are the essential capabilities required for a useful v1 spec. On that front, we also need to consider performance. If we end up with a spec that requires polling and has restricted replication capabilities, it's going to limit its use for many real world use cases and implementers will seriously consider using non-compliant solutions. As it stands, the spec as it's currently shaping risks having limited utility and will require many non-compliant "add ons" to make it usable for many implementers. With v1 of the spec being a long way off I'm not sure we're positioning ourselves for long term success. At the very least, I agree with Michael's suggestion of breaking the spec up and modularising it's capabilities to provide greater flexibility for implementers in the future. Cheers, Chris On Fri, 26 Feb 2021 at 06:05, Dmitri Zagidulin <dzagidulin@...> wrote:
|
|
|
|
Chris
Thanks Manu for the thoughtful reply. Yes, this is a next step. Ideally, we'd map every feature in the spec today to Once you document it (not just the list of requirements we've been going over Sure, I'm happy to make a first pass on this. Any thoughts on the best format / structure? Zooming out a bit to look at the big picture... thinking about how SQL was Totally agree, standardizing SQL is a perfect example. To replay your point back to you, but in a different context: To clarify, I'm not arguing we don't need a standard. To work with the same analogy -- if we're building an SQL standard we should be looking very closely at MariaDB (and other prior art / SQL-like implementations) when developing that standard. The process of standards are not to innovate (at least, not primarily). It's Yes, I agree with that. I guess one of my concerns relates to feeling like we're looking closely enough at what's out there, but as noted above I'm happy to make a start on that with CouchDB. Michael has mentioned some other prior-art (in the replication context) that could be reviewed here. I'm sure there's more and across other contexts? Is there other work to document all the prior art that I have missed? The point of meeting 80% of use cases is important, but there's some subtlety in that. Is it 80% of the "total use cases" or 80% of the "most important use cases" -- those are two very different things. And are we dropping use cases because they're "too hard with the current spec implementation" or because "they're not that important"? Regardless, the ability to have a framework of modularising or extending the capabilities seems like something we should seriously consider. That will minimise the risk of developers ignoring the spec because they're locked in to only working with what's provided out of the box. > I understand CouchDB is not a specification, but as an implementation it's Totally agree :) |
|
|
|
RE: Michael has mentioned some other prior-art (in the replication context) that could be reviewed here. I'm sure there's more and across other contexts? Is there other work to document all the prior art that I have missed?
A few weeks ago I highlighted that I was working on a whitepaper to document and diagram the architectural variations for the 4 key layers in a confidential/trust content storage solution. I hope to have a first complete draft “later this week”. You can find a link to a copy of today’s snapshot at the bottom of this email.
The Preface explains the twists and turns this document has taken and hence, why it has the title that it has… Context
“Sometimes called “reasoning from first principles,” the idea is to break down complicated problems into basic elements and then reassemble them from the ground up. It's one of the best ways to learn to think for yourself, unlock your creative potential, and move from linear to non-linear results.” [First Principles: The Building Blocks of True Knowledge(https://fs.blog/2018/04/first-principles/)] “I think it is most important to reason from first principles rather than by analogy. One of the ways we conduct our lives is we reason by analogy. We do this because something was like something else that was done or it was like what other people were doing. It’s mentally easier to reason by analogy rather than from first principles.” [First Principles Method Explained by Elon Musk (https://www.youtube.com/watch?v=NV3sBlRgzTI)]
PrefaceThis is a first-principles software architecture reference model (ARM) whitepaper that documents the comprehensive analysis of the potential architecture variations for the Trust Content Storage Stack (TCS Stack) including detailed interim as well as final assessments and recommendations.
The intent and purpose of this document has taken several strategic turns during the course of its development. Originally entitled Secure Data Storage Working Group (sds-wg) Confidential Storage (CS): Functional Architecture Reference Models (CS-FARMs), the initial goal was to help evolve the SDS Layers diagram in the Confidential Storage 1.0 Specification (https://identity.foundation/confidential-storage/#ecosystem-overview) to make it more useful for discussing the technology and architectural options for supported replication between encrypted data vaults.
This whitepaper evolved quickly over the subsequent weeks from being a few pages in length to more than 200 pages. The whitepaper had forked from its original mission. The purpose of the current version of the whitepaper to fully describe the architecture variations for a new project: Trusted Content Storage Architecture (TCS Stack).
The work documented here was performed under the auspices of the Trusted Digital Web project in the Hyperonomy Digital Identity Lab of Parallelspace Corporation.
NOTE: The publication of this document coincides with recent discussions (January-February 2021) about secure data storage solutions in the Decentralized Identity Foundation (DIF) Secure Data Storage working group (sds-wg) that were taking place during the development of the Confidential Storage specification. This is not a DIF publication, unofficial, official, or otherwise.
Here’s a commenter link to a copy of today’s snapshot: https://drive.google.com/file/d/1-Ve263KT7svN7oppIHyhK6idXikcbBsn/view?usp=sharing
Any and all feedback is greatly appreciated.
Best regards, Michael Herman Sovrin Foundation Self-Sovereignist
Self-Sovereign Blockchain Architect Trusted Digital Web Hyperonomy Digital Identity Lab Parallelspace Corporation
From: sds-wg@... <sds-wg@...>
On Behalf Of Chris
Sent: March 2, 2021 8:38 PM To: Manu Sporny <msporny@...> Cc: sds-wg@...; sds-wg@dif.groups.io; Credentials Community Group <public-credentials@...> Subject: Re: [sds-wg] Reminder and Agenda for Confidential Storage Spec Call - Feb 25 2021
Thanks Manu for the thoughtful reply.
Sure, I'm happy to make a first pass on this. Any thoughts on the best format / structure?
Totally agree, standardizing SQL is a perfect example.
To clarify, I'm not arguing we don't need a standard. To work with the same analogy -- if we're building an SQL standard we should be looking very closely at MariaDB (and other prior art / SQL-like implementations) when developing that standard.
Yes, I agree with that.
I guess one of my concerns relates to feeling like we're looking closely enough at what's out there, but as noted above I'm happy to make a start on that with CouchDB. Michael has mentioned some other prior-art (in the replication context) that could be reviewed here. I'm sure there's more and across other contexts? Is there other work to document all the prior art that I have missed?
The point of meeting 80% of use cases is important, but there's some subtlety in that. Is it 80% of the "total use cases" or 80% of the "most important use cases" -- those are two very different things. And are we dropping use cases because they're "too hard with the current spec implementation" or because "they're not that important"?
Regardless, the ability to have a framework of modularising or extending the capabilities seems like something we should seriously consider. That will minimise the risk of developers ignoring the spec because they're locked in to only working with what's provided out of the box.
Totally agree :) |
|
|
|
Manu Sporny
On 3/2/21 10:37 PM, Chris Were wrote:
Sure, I'm happy to make a first pass on this. Any thoughts on the bestGreat! Thanks Chris. I suggest Google Docs to start -- that'll help everyone collaborate using tooling most everyone is used to and provide feedback. The structure is up to you, of course. I would suggest that you try listing all the features of EDVs and how CouchDB maps to those features. You might also want to take the opportunity to highlight the features that you feel are important. To clarify, I'm not arguing we don't need a standard. To work with the sameYes, agreed. :) The point of meeting 80% of use cases is important, but there's someThat's an excellent question. I expect it's mostly the latter, but it may be coming across as the former. Typically, when groups can't agree, they inject optionality into the specification... which exchanges the short term problem of agreeing on one solution with the longer term problem of interop challenges. That said, extensible specifications tend to last longer than rigid ones. It's a trade-off, like many other engineering decisions. :) In any case, sounds like there is alignment on the next step (and the general philosophy), which is good. :) -- manu -- Manu Sporny - https://www.linkedin.com/in/manusporny/ Founder/CEO - Digital Bazaar, Inc. blog: Veres One Decentralized Identifier Blockchain Launches https://tinyurl.com/veres-one-launches |
|
|
|
Here’s a better link (I think): https://docs.google.com/document/d/1-Ve263KT7svN7oppIHyhK6idXikcbBsn/edit
From: Michael Herman (Trusted Digital Web) <mwherman@...>
Sent: March 3, 2021 1:46 AM To: sds-wg@...; Manu Sporny <msporny@...>; Chris <chris@...> Cc: sds-wg@dif.groups.io; Credentials Community Group <public-credentials@...> Subject: RE: [sds-wg] Reminder and Agenda for Confidential Storage Spec Call - Feb 25 2021
RE: Michael has mentioned some other prior-art (in the replication context) that could be reviewed here. I'm sure there's more and across other contexts? Is there other work to document all the prior art that I have missed?
A few weeks ago I highlighted that I was working on a whitepaper to document and diagram the architectural variations for the 4 key layers in a confidential/trust content storage solution. I hope to have a first complete draft “later this week”. You can find a link to a copy of today’s snapshot at the bottom of this email.
The Preface explains the twists and turns this document has taken and hence, why it has the title that it has… Context
“Sometimes called “reasoning from first principles,” the idea is to break down complicated problems into basic elements and then reassemble them from the ground up. It's one of the best ways to learn to think for yourself, unlock your creative potential, and move from linear to non-linear results.” [First Principles: The Building Blocks of True Knowledge(https://fs.blog/2018/04/first-principles/)] “I think it is most important to reason from first principles rather than by analogy. One of the ways we conduct our lives is we reason by analogy. We do this because something was like something else that was done or it was like what other people were doing. It’s mentally easier to reason by analogy rather than from first principles.” [First Principles Method Explained by Elon Musk (https://www.youtube.com/watch?v=NV3sBlRgzTI)]
PrefaceThis is a first-principles software architecture reference model (ARM) whitepaper that documents the comprehensive analysis of the potential architecture variations for the Trust Content Storage Stack (TCS Stack) including detailed interim as well as final assessments and recommendations.
The intent and purpose of this document has taken several strategic turns during the course of its development. Originally entitled Secure Data Storage Working Group (sds-wg) Confidential Storage (CS): Functional Architecture Reference Models (CS-FARMs), the initial goal was to help evolve the SDS Layers diagram in the Confidential Storage 1.0 Specification (https://identity.foundation/confidential-storage/#ecosystem-overview) to make it more useful for discussing the technology and architectural options for supported replication between encrypted data vaults.
This whitepaper evolved quickly over the subsequent weeks from being a few pages in length to more than 200 pages. The whitepaper had forked from its original mission. The purpose of the current version of the whitepaper to fully describe the architecture variations for a new project: Trusted Content Storage Architecture (TCS Stack).
The work documented here was performed under the auspices of the Trusted Digital Web project in the Hyperonomy Digital Identity Lab of Parallelspace Corporation.
NOTE: The publication of this document coincides with recent discussions (January-February 2021) about secure data storage solutions in the Decentralized Identity Foundation (DIF) Secure Data Storage working group (sds-wg) that were taking place during the development of the Confidential Storage specification. This is not a DIF publication, unofficial, official, or otherwise.
Here’s a commenter link to a copy of today’s snapshot: https://drive.google.com/file/d/1-Ve263KT7svN7oppIHyhK6idXikcbBsn/view?usp=sharing
Any and all feedback is greatly appreciated.
Best regards, Michael Herman Sovrin Foundation Self-Sovereignist
Self-Sovereign Blockchain Architect Trusted Digital Web Hyperonomy Digital Identity Lab Parallelspace Corporation
From:
sds-wg@... <sds-wg@...>
On Behalf Of Chris
Thanks Manu for the thoughtful reply.
Sure, I'm happy to make a first pass on this. Any thoughts on the best format / structure?
Totally agree, standardizing SQL is a perfect example.
To clarify, I'm not arguing we don't need a standard. To work with the same analogy -- if we're building an SQL standard we should be looking very closely at MariaDB (and other prior art / SQL-like implementations) when developing that standard.
Yes, I agree with that.
I guess one of my concerns relates to feeling like we're looking closely enough at what's out there, but as noted above I'm happy to make a start on that with CouchDB. Michael has mentioned some other prior-art (in the replication context) that could be reviewed here. I'm sure there's more and across other contexts? Is there other work to document all the prior art that I have missed?
The point of meeting 80% of use cases is important, but there's some subtlety in that. Is it 80% of the "total use cases" or 80% of the "most important use cases" -- those are two very different things. And are we dropping use cases because they're "too hard with the current spec implementation" or because "they're not that important"?
Regardless, the ability to have a framework of modularising or extending the capabilities seems like something we should seriously consider. That will minimise the risk of developers ignoring the spec because they're locked in to only working with what's provided out of the box.
Totally agree :) |
|
|
