Welcome to today's episode of solar gas!
It is with a bubbly excitement that I will be introducing KDM from New
Design Congress and we just had a wonderful conversation and it gets into
a bit of the high level like how do you actually design securely and for local
first and peer-to-peer protocols and you could think of this episode as a
primer since this is probably the first of a little bit of a series and where we
introduce a Kade and New Design Congress framework as well as the thinking of
security in peer-to-peer and local first networks from a socio-technical
perspective. So with no further ado, let's come Kade! I'm so happy to be having you
here and we've been chatting away for quite a bit already and there's so much
to cover. But first and foremost one of the things we talked about earlier and
honest, you have so many world records. Specifically you have a world record in
swimming. You have three? What? Oh my god yeah. That's incredible! Three! Yeah, three
events, three events like you know you can do different strokes and length
distances and stuff. Just casually drop that. It's what's known as S6
swimming which is a classification in Paralympic sport. 200 meter individual meds,
sorry excuse me, 200 meter individual meds, 400 meter freestyle for short course.
That's great! And I feel like this also sets the basis kind of for like your life in
some form because you've had quite very interesting and which ones say like
a substantial life and also your perspectives are quite substantial. So if we go back a little bit
where did this all start? Like how did you end up being so interesting? Goddamn it!
So I think I would preface this by saying so I have osteoporosis and I was born with it as a
specific kind called Spondyloepfisile dysplasia which if you know the actor Warren Davis, I think
Warwick Davis, Warwick Davis. He's a short statured person he's about. He's quite short but he's been
in a number of really successful really famous films. He's a Hollywood actor. He's in I think
Harry Potter and a number of things like that. I actually don't follow this guy's career. It's
just somebody who has the same kind of osteoporosis I have. But what happens when you have something
like that is you it's the systems that exist around us will try to classify and read you in
very specific ways and often get it really wrong. You would call that in modern terminology like
ableism but I think it's deeper than that. I think there's an institutional desire to classify
people in different ways and it's universal and when you have something that is so out far of the
accepted norm then it becomes very very clear to see that. You have to fight against it for
your entire life. So this is where I think my sensitivity towards understanding
digital systems comes from because these are downstream. We use digital systems in order to
manage the way we communicate how we represent people and ourselves and how we store information
about ourselves and other people and that I think is intrinsically related to how we classify
people outside of that before the cybernetic digestion before we turn people from people and
attributes into representations. So downstream from that you mentioned the Paralympic swimming.
I guess the turning point the catalyst which we really drove that move into doing lots of
different things is I was trialing for the Paralympics as a teenager is about 19 years old
and this is after I'd achieved the world records and in the 200 meter individual medley
race that I was in the qualifiers for I did a turn the first turn from butterfly to backstroke
and as I pushed from the wall my hips disintegrated from the osteoporosis and I ended up
instead of going to the Paralympics I ended up getting a double hit replacement at 19 years old
and in the recovery from that it kind of wiped out my swimming career but it also made me realize
that I had to do like probably two things a little bit fatalistic one is that I was probably
living on borrowed time which luckily has turned out to not be true and the second is that that I
had to do a lot of different things and put like myself into lots of different places rather than
banking all on one one kind of thing so fast forward to 2015 I end up working accidentally on
signal in version one as part of the team that helped launch signal the secure messaging app
and get taken to Hawaii after volunteering on the github project for six months and then
flown to Hawaii and actually work on signal directly and I end up having a rather large
disagreement with Moxie Marlon's bike the the leader of open whisper systems and the
co-founder of signal over the use of phone numbers as an identifier which has become a huge issue
with signal which has been weird to kind of have been the quiet I won't even call it the source
because I never talked about it again I lost that argument obviously and that kind of really
blew me away I wrote an essay last year and was due to circumstances beyond my control only able
to publish it in March of this year but on our website newdesigncongress.org you'll find the
website they find an essay called who will remember us when the servers go dark which chronicles the
experience that I had that I would call if you like my origin story the second half of the origin story
in which I get a sense of the violence at the heart of the digital world and how it relates to
historically other forms of colonialist violence and imperialist violence
and not to say that this is not to say that Moxie Marlon's bike is an instigate of him that
but rather that like the technology lives downstream from these these goals and that it will it's
desire above all is scale and it will do that regardless of the cost and that was I believe when I
started to within I guess three years of that I would have moved to Berlin joined Tactical Tech
for a few years the data privacy non-profit and then by 2018 I would have formed new design
Congress which is the research organization that helps to understand the gap between what is said
to be happening and what is actually happening in digitized societies and then by 2026 now I've
got the research and development lab called Para Real Limited and so over the past nearly 10 years
now of this work we've been working very closely on understanding different ways in which
different I would say first principles if you like of digitization and how they relate to the real
world and those include digital identity and how we represent ourselves how we communicate with
each other over the network and what the the material costs are of all of this stuff and this is
the entanglement I think between those three things is what covers a lot of the kind of work that we do
and I feel like this example that you brought up or it's not an example it was very real and still
is very real with signal and the phone numbers is kind of a great case example to start from when
one's trying to understand the approach you take towards security and it's it's I was thinking
as your essay this is fine points out very acutely it's a missing perspective in a lot of projects
especially projects that aim to take privacy seriously or aim to be some sort of ethical or
utopian solution and so I'm thinking if we look more closely here because I most people are familiar
with maybe pen testing or testing for security vulnerabilities or using encryption you know but
like these very technically focused ways of ensuring security and I think this also ties into
the whole conversation of like different data systems trying to solve trust technically rather
than solving trust socially and but your perspective on security it shifts the lens quite strongly
and your essay kind of was a wake-up call for an entire movement you mentioned a little bit before
but like the history of like what what's been going on what's happening since that essay and
what's been what's what's emerged as like paradigm shifts one could say okay around about
July of 2020 so you know not to bring us back to that point but middle of lockdown where a couple
months into the pandemic a couple months into new design congress as a formal full-time
piece of work rather than a side project and I'd written previously my first I would say new
design congress piece called on weaponized design at tactical tech back in 2018 and your design
congress is your company if the research it's the research organization yeah as I mentioned a
little bit earlier so it's a research organization that we run that is building a body of work to
confront the gap between what is said to be happening and actually happening in digitized
societies so it's it's really what you're sort of taught what you described just then about like
the difference between um the belief that engaging only in device based or systems based security
brings safety and what is the actual outcome of it as like an example of the gap between
the two right that we that we look at very closely yeah so I wrote this essay in 2018
called on weaponized design which described how it was impossible to have a system it was possible
excuse me it was possible to have a system that could harm users without actually breaking in any
way and without really design it it systems or interfaces that harm users whilst performing
exactly as intended and these are the facebook news feed for example back in 2018 this is a
very quaint example now of being able to produce an emotional contagion in users through like
viral posts is an example of weaponized and the entire thing is operating precisely as is intended
to be but it can whether or not the designers are aware of it or not it can produce emotional
responses that are you know deeply harmful and we knew this as early as 2016-2017 when people
started doing research into the effects of the facebook news feed and its algorithms um from
that a number of other it's like one of the genesis pieces of new design congress and from
that there's a number of ideas that descend on that from that one of them is an essay I wrote
specifically to warn one of my beloved spaces even though i'm a little bit of an outsider
the pia to pia community so this i wrote this essay called in 2020 called this is fine optimism
and emergency in the pia to pia network in which i describe how the last 15 years had seen a surge
of interest in decentralized technology from blockchain projects through to that scuttle
butt activity pub and that after this period of time which kind of ended when bit torrent died out
and when the streaming services began like when Netflix began to take off i guess kind of the
development of the iphone and the move from desktop computers into smartphones which for years
couldn't really do decentralized tech very well um there's not you know what i could see was a
renaissance we could all see it at the time a renaissance of interest in decentralized technologies
that had explosive growth driven by the desire for platform commons and community self-determination
and what i say in the piece is that the goals that we had collectively back then are fundamentally
at odds with and a response to the incumbent platforms of social media cultural distribution
data storage and and so on and the piece was a warning that by the 2020s centralized power
and decentralized communities are on the verge of outright conflict for the control of digital
society and that the resilience of centralized networks and their political organizations
remained significantly underestimated by people in the peer-to-peer space and uh on the flip side
the decentralized networks and the communities that they serve have never been more vulnerable
um again this is written a few it was written a year before or actually no it's written at the
end of the first trump administration when a lot of people were beginning to feel a little bit more
relaxed that something better was coming after the first trump administration
and what i could see coming new space on the work they've been doing was like not that at all and
so my concern was that you could look historically and see what happened to decentralized communities
in previous decades you could then like apply that same thing because you know none of the issues
had been solved and as a result the peer-to-peer community was dangerously under-prepared for a
crisis-fueled future that has and i say in the essay very suddenly arrived at our door
and uh six years later that turned out to be true you know um and that's also like it's something
like that a lot of these uh older like now i do you scuttle about as an older network for these
precise reasons because they cannot hold up to actual security risks in real life and are not
therefore not scalable beyond other reasons of not being scalable so it's really taking a shift
and i think a lot of these newer protocols like peer-to-pumba have started embedding these
right this and also especially willow um has started embedding these thinking into their
methodologies and design of protocols which is also like stems back to this colonialist like
issues of who is actually building this technology right not necessarily always the people who are
at risk so this kind of thinking has started taking form in people's realization of building
these projects are you seeing that or is there still like a large discrepancy between security
in reality and security in theory that's a big question with lots of answers yeah
the starting the starting is that i hear you very clearly on the secure scuttle
about thing and i feel kind of the same way what was interesting is that when i released the piece
and i have had a some time to reflect on this and i know that part of this has to do with the
resistance to the ideas of the essay and part of it has to do with the antagonism that we have
at new design congress and it was a relatively new organization at that point um that piece
really divided people and i went from someone who had co-organized you know the first peer-to-peer
web spaces in Berlin like the very first one that um Lewis Center and um couple other people
put together at Trust back in like 2019-2018 um i was sort of going from being very participatory
in this sort of space to really making quite a few uh and i wouldn't say enemies but like people
who really didn't like what i had said and didn't want anything to do with me as a result of that
um and i named a number of projects this is something that i do because i'm not really i
think we would sort of run out of time but i named projects right i didn't say that people
were bad people but i named projects right and one of them of course was the secure scuttle
butt which i described as a what did i describe it as like it's a mercultry of forensics investigators
dream come true um where everything because everything every piece of um information
is permanently etched into a record whilst that is an absolutely elegant offline first system it also
uh given the fact as well that it was hosting a left-wing anti-capitalist political movement
was like precisely not what you wanted to have if anybody if anybody like in an
estate security service or something took the the threat of that seriously right it was a very very
it's unbelievable that nothing seriously bad happened during that time frame it was very scary
but but what i've noticed over the past especially the past three years so a year later i put out
with incense witch ray mccellvey benjamin roya christson day uh myself and peter we put together
and based off of an idea that i had about how we could start defending against these issues
um we put together identity primitive called back channel we started from this idea that
one of the threats in digital systems especially decentralized digital systems was that social
engineering attack was particularly effective where you could um pretend to be somebody on a
network and it didn't matter how much encryption or how well designed your system was um if someone
could convince you in the network in the peter pia network that they were someone else you know
equivalent of like catfishing someone on tinder but for purposes other than messing with people in
dating service something more you know it sounds side note yeah fun side note there we actually did
that on scottabut just as a joke because you know scottabots full of jokes it was more like a
humorous thing but like someone just created an account and it was four people and we had all
agreed to do this joke and it was just a joke um and we made an account called uh
dominic and dominic was the founder of scottabu and then everyone else started
verifying that dominic was dominic oh my god i didn't know the story
it was me kegs and a few others um and everyone else started verifying that dominic was dominic
but then dominic the real dominic was like no i'm actually i'm not gonna lose my account
and it was exactly what you're talking about here that like kind of co-opting someone's identity
in this case i find that like it's quite telling that we had to be five people who were friends
of dominic to agree to co-conspire against dominic but the the case stands that like the
digital identity of dominic who had made the software that we were creating a digital identity of him
on top of was having to fight us saying that we were not him i love that story it's so it's
lucky that he had those five rich friends and not like his worst enemies
it's yeah but like i guess the that's the thing that made it credible that we were actually his
friends because i guess if some random account came on and said this is dominic i guess it would
be because it's a social graph i i don't know i would love i would love your take on this like
because it's the social graph of trust verifying the identity does it become more or less safe
like in that sense i i don't think it's a case by case basis right and it depends upon
a number of factors and some of which are outside of the summer which are really temporal like
if dominic had been a way on holiday for example and away from his computer or you know sailing or
something or or just completely offline um it could be possible that someone who had been like
more closely tracking his movements with the goal of you know impersonating him for some reason
for a gain um could have used the time when he was offline to build a quicker version of trust by
like reappearing as a new account and then convincing certain people to sign that and then
trying to build that five person team that you're talking about but through convincing other people
that this attacker was him so you're right in the sense that like one of the web of trust is
is like what you've done is you've like speedrun the web of trust problem um where the five of you
because you were conspiring together and because you were friends with dominic that made it very
easy to do very quickly but you also not i i assume none of you were particularly skilled in like
actual adversarial like social engineering attacks like not that you have to be trained in it some
people are just naturally better at it than others as well as people who practice but
it's not like you know if you're doing it as a joke the stakes alert the the intent and the
motivation is a bit lower so that all plays in your favor the fact that you had the five people here but
it's i i would say it shows that it can be difficult it doesn't show that the five people
themselves protect you what it does show is the power of the social graph itself and the power of
of relationships that have existed outside of the network itself right that's where the real
power of social engineering exists and so gosh i wish i'd known that during the that period of
time i would have absolutely used that in the essay you would not believe god damn
no no no no no no no people would have done that for sure um but
i think we yeah i mean everyone has got about like we were utopian like trying to just strive
to build what we believed could be a better world and i think there's this yin and yang relation
between people who have your like gift of critiquing and and breaking things down to like the small
pieces and also seeing the holistic aspect at the same time i don't want to say penetrate but like
to to pierce the veil of like this utopia bubble but i think both are needed like we need to be able
to to that's another sidetrack which i would love actually to hear your thoughts on this but
just to finish the thought um like we need to have both the critical and the the visionary
together in order to actually have the forward momentum that's sustainable and i think a lot of
the visionaries in scuttle but uh want to not everyone because you got a lot of angry people
coming at you after this article but i think a lot of them want to actually understand how to
move forward uh sustainably and safely um but yeah so i think i agree with you i think the
difference where we would sit is that i i would call it aspirational uh in a sense because uh and
this is a i don't want to open a can of worms here but we could definitely talk about this
route is i actually kind of believe based on how humans have developed as a species um and again
with keeping in mind the wide experiences that individuals have and that cultural differences
between different um parts of our you know global society if you like not to use a again a really
seriously colonizing to a colonialist term the the human like humans are very sensitive to
scarcity it's one of the things i think that drives a lot of fear um i think that scarcity is one
of the things that's leveraged to uh weaponized populations into excluding and othering different
parts of society um and and so one of the tension points that i hold in my head is that utopianism
promises things that i think humans are for the most part somehow i want to say biologically unable
to accept but somehow intrinsically resistant to the and unable to really fully grapple with the
concept of of of abundance in like a serious way and i think you can see this very clearly in
examples such as like the modern the modern condition right where um in some ways there's
abundance like um for all of its deep flaws the number of people you know whenever you point
out the the like the crimes of capitalism you'll always have economists and other people defenders
come forward and say that capitalism has lifted the largest groups of people out of poverty now
sending aside that as an argument um uh that can be attacked from multiple different ways
one thing i would say to that is that i think it's interesting that even if you accept the very
like a very narrow definition of that claim or very narrow um interpretation of that claim that
there are fewer people in poverty in capitalist societies than they were historically before
they were industrialized even if you um interpret that as in its narrowest um version of that claim
um people we still have the society still struggles over the same issues of wealth accumulation
of fear of um resource fear you know cost of living things like that and without again
with keeping this very high level so i don't we don't veer off into another direct dimension like i do
worry that that utopianism as a as an aspirational goal creates a tension point between like humans
and that goal now sorry to be off topic that no i invited for the question i'm actually
personally really curious about this so i i've been thinking about asking you for a long time so
i'm glad it came up oh i appreciate that so i i but i'm not this is not nihilism speaking right what
i'm saying is that like i'm not at by any stretch of imagination analyst um what i'm trying to say is
that and this is where i do start to call on where i start to call on like some of the stuff that
happened after the essay um i believe you can be visionary or aspirational and somebody who has
a deep critique and the last five years has proven that um the the choices that certain people made
after the this is fine essay was published and after we completed the back general work for example
there are people who were inspired by that work or people who had parallel ideas that they committed
to and started building in that environment as that that world collectively opened up and that
dialogue started to happen properly and then there are people who didn't and what i find interesting
is that we tend to try to separate the visionary from the critic and i think if there was one thing
that i would say that was intrinsic to the initial blowback that i got from the essay and the warning
was that people the visionary for a number of reasons is very protective of the thing that
they're working on and i think that if you're a builder i it took to the detriment of of of of
that work to the to the detriment of all else like and and usually this this this manifests in the
sense where the the visionary will accept criticism on the terms in which they are developing their
this vision on so in in the case of decentralized systems this will be people accepting github issues
but not fundamental or philosophical critique of the thing that they're building in the first place
right which is but both of these are important right like the tech stack you choose is just as
important as who you choose to be excluded from the system that you're designing itself like
all of these ultimately are legitimate questions and you can't if you anoint yourself as the builder
if you want to inherit the arrogance of the builder of the vision visionary then you have to accept
the arrogance of the critic who believes that they can come and tell you why the thing that you're
building poses a threat and i would say especially not to invoke the kind of idea of ableism here
because i do keep this that side of me quite quite private um from expertise that can see it coming
right because it's expertise that's lived in that world in in a in a sense in exile from the norm
for you know my entire life in this case now of course like online much harder to tell i don't
identify with my osteoporosis online things like that but there were people who knew me i had people
who had seen me and and spoken to me in real life or seen talks i'd given and sort of you know there's
a sense of of of of like the how do i put this it makes me sound like i'm still really angry about
that this is fine piece and in some ways i think i might be a little bit um because i feel like
there's i'm sorry and that's okay yeah well i'm angry not because of the blurb i'm angry because
i feel like we missed there like there are ways in which things could be even better than they are
now in certain ways like the improvements the kind of consolidation around um decentralized
digital identities that are much more carefully designed than they were historically or the
the promotion of end-to-end encryption these sorts of things have been really powerful
but i feel like there was so much time lost in such a terrible lull it like in a lull in the
terribleness of the world that ah man if there was a way in which i could have written this better
or if there was anything i could have done to communicate the ideas better maybe that's not
that offering or just a part of a bigger system and what you did had a huge catalyzing effect on
larger pewter pew and local first ecosystem space and yeah to finalize that thought is that like
i believe that the visionary and the critic can be the same person and i think that to have someone
come at you and offer substantial criticism even if it's confrontational or antagonistic
if the criticism itself is deeply thought and something more than what you should do is x or y
or something that very clearly pushes a motivation that doesn't align or is an uneducated opinion
then i think you can kind of it's in i think it is also on you to meet that person where that
where they are in terms of that criticism and i feel like the people who are going to emerge
in the next couple of years as the leaders of decentralized movements because they they are and
will they will be and are being attacked right now um by the centralize um in common as i
described in the essay the people who emerge from that as the wind as the kind of systems that
are resilient to these sorts of attacks are the people i feel who have internalized their ability
as a visionary or as a sort of a leader of these kinds of systems and also someone who's internalized
the ability to critique it at an existential level um and that i think is like i think that
people who are unable to do that read really in danger um themselves their teams and communities
around them when you are completely unable to separate yourself from the vision and consider
the critique especially strong critique foundational critique as an indictment of your vision itself
what does one do when like because some of this foundational critique and i think what causes
some people and it just to sidetrack a little bit more because we have some time before we dive
into the next part which is also very juicy because a lot of people who are met with this
kind of critique especially when it's larger peer-to-peer projects that they've been working years on
and then they come to realize that there's like a foundational issue with the infrastructure and
how it's built and like solving these foundational critiques would mean basically starting over
from scratch like is it because because i think that's what kind of makes people
um bite down and i guess i guess what i'm leading on towards is that like in a conversation i had
a few years ago with Anyosha one of the approaches and one of the reasons that it was so difficult
for especially Skudel but to kind of swallow this critique is because we were built in such a
monolithic way or Skudel but was built in such a monolithic way that it was very difficult to
change small pieces here and there without changing the entire system and then Anyosha had
brought up this perspective that like in order to be more adaptable in order to be more resilient
in order to be more future-proof we need to change the way we build to become more modular
which i guess in turn also becomes an answer to the question i posed unintentionally so but
that we can respond to critique in a better way i'm just guessing because it's easier to change
yeah firstly shout out to the willow team sammy and um alio for being the urges of
recognizing that this is fine critique and kind of baking this entire philosophy into along with
cinnamon when they were um still with us um and when willow was called um earth star
that is a group of people descendant from secure Skudel but who like were even grappling it at
the time before the essay came out i think one of the first times we did a reading of the essay
itself was actually in their discord server in the earth star discord server which was like
only a few weeks after i published um you're right in the sense what what when you refer to
what alio she was saying in that conversation of of needing to produce a way of designing systems
it's way more modular i think that's 100 true i also want to say there's two other things to
and we can touch on this either now or in the future sometime because it's a big issue
that we have to change how things are funded for starters um part of the concern that there's
i talked about ego and fragility of the self as being a big part of the pushback to criticism
but you're also right that there are other reasons why people get feel really vulnerable why
team members who are building projects like this get really feel really vulnerable about
these kinds of critiques especially fundamental ones the first thing i'll say on that is that like
no matter what the nature of a digital system and the nature of compressing the variety of human
lived experience and the kind of material world around us into a digital system means that you
always make a trade-off and it's always going to be bad right there's always going to be a serious
existential drawback to what you're what you're building the question that you ask yourself
that you're as a designer as a like a protocol developer or platform developer as somebody who's
designing these systems is can i live with this and does this align with my politics completely
how closely does it get to how i position myself in the world and what i put forth as my politics
if you're a protocol designer if you're working in decentralized spaces and you're producing
you're engineering some kind of future you are essentially in one sense existing in this kind of
what i would call like a power real state where you are operating both in the digital world and
in the material world simultaneously in a kind of third space and you're creating that third space
so that others can be there as well right in a sense a little bit esoteric but it's really about
these creating these moments of charged sort of social interactivity that you that are mediated
through a ideally controlless decentralized system that's almost kind of like political writing
it's like writing interventions in the form of essays but your essays here occurred and rather
than reading them people are participating in them right and so and so beyond like beyond the
sensitivity towards that the two things i would say is one yeah absolutely what alia would
i talk with alia should pretty frequently i think that that's one of the core things that
they've impressed on me too and there's some stuff that we've been working on which we could
talk about sometime later um that really embodies this completely and then secondly that we have to
have funding we have to have a cultural change in the funding landscape on what the expectations
that funders have on decentralized systems because right now there is no margin for error
and error is seen as something that's has to be defended against or justified rather than
being seen as for what it is which is like an opportunity to to build upon
um and hopefully avoid you know systemic failures of the previous implementation
now of course i understand the risk like funders are this way as well because otherwise you end
up with people spinning their wheels reinventing things over and over again you know like reproducing
the same material every six weeks because they find a flaw in it and they have to start over like
the idea here is not to say this is why it preface this by saying that the questions that you ask
yourself are whether it's politically in alignment and whether you can sleep at night
um the idea here isn't to be perfect but rather that the the the work has to be
defensible not on terms of the mistakes that you've made or the or the the the blind spots that
have been coded into the into the work but rather whether or not it aligns with the politics of
yourself and to a lesser extent but still importantly the politics of the funder and then
building from that the other thing we desperately need are representatives in places like the
wc3 and other standards bodies who are um essentially pooling for the shed consensus on where we feel
these things should stand and what their standards should uphold and actually advocating for those
in the larger society because that's the only way as well that we're going to establish ourselves
for like longer-term strategies around funding around implementation around protecting ourselves
if we if we have that representation as well
and there's a whole world we're going to dive into which is like how does one get a consensus
in these spaces and how do we actually approach this and I think this also boils down a little bit
to this uh paradigm shift that kind of uh came about around 2020 and around the time of your
article um but I think one missing aspect in this conversation that we haven't quite touched about
or touched on which is we've been kind of talking around it but the methodology that you have
developed which mentioned it like three times everyone on the like listening is like what are
you talking about yeah and I guess that makes sense because it's often like the outcomes that are
relevant for people and but there's actually a process that you have charted out and kind of
created in order to discover these kind of fundamental challenges and safety concerns of
socio-technical systems yeah so you you have mentioned it to me and first time I saw it I was
what's axillic so it's an acronym there's two yeah anxiety and axillic yeah yeah yes there we go
and could you could you like help us like if if someone is coming into this and they want to start
understanding things from a social material security perspective where did they start okay
so to understand why something like this needs to exist we start first with what it's in response
to so maybe some people who are listening to this will well know this will be old news then but
there will be people here listening to this that don't know threat modeling is the idea of looking
at a potential looking at a set of conditions that you have right now in a digital system
and then trying to forecast in different ways focusing on different parts of that system like
the computer the network the inputs of like data things like that where an attacker might come in
and actively attack that um so things like um you know the idea of HTTP unencrypted
um web browser communication versus uh HTTPS uh the threat models around why you would implement
HTTPS is that the uh that the the data coming over the protocol in an HTTP request you know
such as your bank information or other data that you like might be sensitive that you've sent to
a website can be listened to when it's unencrypted by anybody between you and that server and so
that would be the threat modeling your threat modeling the idea of well if it's unencrypted
then it can be listened to and so out of that you know the simplistic like very simple example is
to then say well we'll exchange encrypted um keys before we then send an encrypted version
of the same requests over the network and the intermediaries can't read the data that you're
sending and receiving from this website it's like a basic example um go ahead can i jump in here
because an example that for me also when i was watching your talk at CCC this year um
you brought up an example that i was like kind of shocked by and it feels in hindsight like i
shouldn't have been but it also gave me a really nice perspective to understand how socio socio
technical threat modeling can also discover non-technical security challenges and you brought
up this case that was i think it was uh southeast Asia and uh there was a bank or some large large
collaboration yeah yeah in Hong Kong that was doing a transfer of large amounts of money and then
as far as i remember it correct correct me if i'm wrong um then there was this uh teller or not
teller but like a person working at the bank or company that was supposed to be transferring
the money and then there was deep fakes in like seven different people um who were representing
his co-workers right who were telling him to go through with this transfer right but it was a
false transfer and it became the highest of how much money uh let me i've got that number there
hang on um so to firstly to answer like to to like um you've got a really good memory that is
almost precisely what it was um the idea here is that it was a Hong Kong based firm 200 million
Hong Kong dollars um it's an uh it's a routine it's a it's a it's a routine phone call in a sense where
this is somebody who was able to who has the authority to transfer large sums of money inside
this large company um he's but the protocol is is that this person has to sit down with the CFO
and other senior team members and get the okay and do like a whole procedure where they talk to
each other and then verbally okay it and then they transfer and the idea here was that this
highest of 200 million Hong Kong dollars was completed because the people who were part of this
sort of security ritual which is meant to be you know face to face or over zoom call whatever
where you're verifying each other based on the presence of the other people um all of those
people involved except for the the target who transferred the money were deep faked um that was
in 2024 just over two years just over a year and a bit ago and um and so so getting to the
difference that that's a really good segue into the difference between sort of threat modeling and
socio-technical threat modeling the problem with threat modeling it's descendant from you know
darker department of defense style uh perspectives on seeing the world where you try to organize
the world into systems that you can understand and you eject anything that doesn't fit this model
that you created like the system that you've created cybernetic system as entropy as something
that um is an excess noise right as a something to be discarded from the system and as a result
of that all of the modern threat modeling practices tend to focus on devices on security systems on
platforms um on networks etc like the the actual digital side of things when in reality um the
consequence of that is that the social side of um the threats that emerge the kind of weaponized
design the the issues that I raise in this is fine and in the all the work they've done uh these
are of course intrinsically linked to the use of digital systems but they are fundamentally not
issues of digital systems they are what's known as socio-technical um uh consequences and social
technical issues around how we like the relationships that we have that are intermediated between
um digital devices and the custodianship of data the political downstream consequences
consequences of social ones etc and so what's emerging now which is very exciting is an entire
field which has been around for a while but is I think beginning to really crystallize is a discipline
within I would sort of say the humanities rather than comp sci around um viewing digital systems
in this lens and so for the last I would say seven years so from like 2015-20-2016
2016 was when I was starting to think openly about how to produce what I would call like a
generative framework that is a system in which you start with a handful of questions that then
emerge into your own taxonomy of of risk um but how do you build a system that is accessible that
you can use in lots of different contexts that de-center the system and instead refocus on
on individuals communities and and the social networks the political networks the economic networks
around these individuals um and still produce a working um understanding of the kinds of threats
that are individually experienced or collectively shared around the use or in at the insertion of
digital systems in these spaces and so I started this work I would say just after my clash with
signal and I had been picked up to become the um chief product designer a chief product officer
at a company called Spyderook which at that point was a very famous um end-to-end encrypted drop
box competitor that Edward Snowden had famously said in an interview was unable to be um uh
cracked by the NSA and that like gave them a wave of attention and and funding so it was the
head there of their of designing and helping the product team put things together with the CEO
and this is where this concept of anxiety came from an anxiety is a socio-technical threat analysis
framework that has seven vectors through which infrastructure projects can produce harm
whether through direct action architectural failure or external appropriation and anxiety is an
acronym or is a seven vectors you've got appropriation the capture of identity data or
infrastructure by a third party negligence unexpected governance failures specific to potential
decisions that a designer may reasonably make this is you know everybody from a protocol designer
to a um an app designer exclusion failures to account for material conditions that temporarily
or permanently block access for individuals communities or entire populations out of the
system or downstream from that other services that rely on that system in personation which is a
social engineering attack vector attacks against infrastructure staff or users or that use the
infrastructure itself to impersonate an attacker exploitation which is the abuse and attacks driven
by system based incentives so these are things like um uh where you have like when you add a
a token to a decentralized network and then you get like fraudsters come along this is like
exploitation where the goal is to like the attacks are accelerated as a result of the financial
incentive toxicity direct harms to the social fabric made possible by the project and yielding
attacks that rely on coerced consent anywhere in the infrastructure so anxiety takes these seven
vectors and um through a series of um pedagogical sort of designed and we could this is another
talk for another day there's a whole set of pedagogy um inspirations that I pulled from as part of
this there's different methodologies within that to kind of tease out in in a participatory way
some of the threats that emerge and how they how they interact with each other and then once you
have an understanding with that you can do this as a practitioner or you can like bring communities
into it and work it sort of flexible enough to do both the kind of response to anxiety is a
design framework that we call axilic like exile and axilic um is is a much more looser system because
it doesn't allow you to um it's not like a prescriptive thing like human-centered design but instead
it has nine core tenets so built for post-cope this is the worst case as the worst case scenario
is your central constraint it's an empathetic interface design that prioritizes cognitive
diversity and intellectual sovereignty relationship-based identity a person is a person through other
people collective access to emerging tech this is things like economic justice and resisting
cloud coercion self-hosted infrastructure by default and it uses um you know video game inspired
accessibility so things like how kids can set up Minecraft servers by themselves and bring their
friends in without having a third party involved um no network by default connectivity is seen as
a liability and it requires explicit permission deletion is fundamental right decentralization
is worthless without it even though deletion is one of the hardest things to do in decentralization
break these are systems that break the frame these are inter-processed integrations anti-siloing
of data and connectivity and um uh designs that exist beyond the electron app window things that
can move through a system and kind of the last tenet would be the end of what we would call what
Fukuyama called the end of history the end of the end of history where we consider permanent
instability as the operating assumption rather than the belief that liberalism had triumphed and the
world was a stable place consequently and that the kind of values that axillic ab once again is
an acronym are that an axillic desert an axillic system is one that is ephemeral with graceful
graceful degradation that allows for decay exits with allows for exit reversibility right to
deletion it is intentional with explicit adversarial modeling and non-weaponizable design it is local
is um speeches infrastructure independence human rebuildable systems and pluralism economic
independence informed as in participants understand and have cognitive agency and consensual in the
sense that participants express their own sovereignty and it allows for explicit permission it demands
explicit permission for participation in the system um and what I would call it based on the
work that we were talking with the more broader work that I do is that it's the act of knowingly
building within the the power reel again this idea there's a system designer you're kind of
building between the real world and the digital and that third space is where a lot of I guess
the ideological struggle plays out today these are the things that we use that have been the
bill in in in the in the background I guess since 2016 um to you know over the last year and a half
I've been more formal like formalizing more directly I'm hoping to actually publish this next week
like the first draft white paper for this stuff I hate using the term white paper the first draft
text if you like all these two systems um yeah I'm really looking for a threat because that was
actually my absolutely next question which was how to learn more about this and where to find it
I have the URL in my head and the URL is um newdesigncongress.org slash en slash pub slash anxiety
dash exilic that will be the URL awesome and I will link it in the notes to this episode yeah and
if that 404 is on you then you just go to new designcongress.org and it'll be on the front page
super duper so I'm thinking right we already before we even started interviewing realized that
there was going to be so much part of this like conversation that we could have that could be had
and that we would like to fit in that we kind of honestly knew yeah exactly we already knew that
we were not gonna fit at all in one episode so I'm thinking brainstorming here uh one potential
next episode which I would be really curious and I would love to experience is tackling this
challenge of digital identity but specifically distributed or decentralized identity absolutely
in uh from the perspective of these frameworks what do you think well we've just there's a whole
story there's a whole backstory for this too but we have been sitting on a report that we weren't
able to publish um on digital identity until very recently so that would be a banger of a starting
point I think um there's a lot as a sneak peek I would say the way this would we would be talking
about a four and a half year long uh study that we conducted on the failures of digital identity
plus the parallel track that was emerging uh both of as a set of the learned experiences of certain
protocol design as in developers combined with the this is fine essay and some of the work that
new design congress did in 2021 with incand switch which either directly inspired or validated for
others um the idea of a different kind of identity primitive and sort of five years later four
and a half years later seeing some of the work that has either directly or in parallel emerged
as a result of that there's an entire collection of digital identity systems distributed identity
systems uh that really I think offer an example of this axillic um methodology that can take into
account some of the issues that exist today that historically hadn't been considered important
we could do in a whole episode on that very easily here amazing I'm already excited in sitting here
jumping in my chair but with that said we will have to leave the rest of that conversation
and keep the listeners on a cliffhanger because this is the wrap for today's episode and
greatly looking for us to the next conversation we will have Kate and is there any last notes
where do we find you how do we connect anything any shout outs you'd like to make oh to the team
new design congress uh this is like Lewis Center and Benjamin Royer and all the people who have
worked with us in the past my lawyers that would be a good one that's for tomorrow's next episode as
well and um yeah I mean I should do the shout out to allie Rish for that like the teams that have
been working on stuff uh over the last few years um and I guess if you wanted to find me
I am at post.lurk.org.com at Sheba Computer S-H-I-B-A computer uh and then yeah you can find our
stuff on newdesigncongress.org and finally if you would like to do some pretty heavy duty and very
very helpful um socio-technical threat modeling uh and analysis of your work come talk to me uh
we can apply some of the anxieties framework stuff with you and get some really really good
actionable outcomes as a result of that kind of work so if you're working on something and you want
to um and you want to really analyze it about how it will work in the world let's talk awesome
thank you so much for joining Kate thank you so much for having me I super appreciate it this
has been really fun yes I think it's the same and I don't have a wonderful rest of your day
with your grumpy little Sheba's right now and yeah I go out yeah they're okay yeah as long as
the rain stops we'll have a great one thank you bye
Joe