New Design Congress

Welcome to today's episode of solar gas!

It is with a bubbly excitement that I will be introducing KDM from New

Design Congress and we just had a wonderful conversation and it gets into

a bit of the high level like how do you actually design securely and for local

first and peer-to-peer protocols and you could think of this episode as a

primer since this is probably the first of a little bit of a series and where we

introduce a Kade and New Design Congress framework as well as the thinking of

security in peer-to-peer and local first networks from a socio-technical

perspective. So with no further ado, let's come Kade! I'm so happy to be having you

here and we've been chatting away for quite a bit already and there's so much

to cover. But first and foremost one of the things we talked about earlier and

honest, you have so many world records. Specifically you have a world record in

swimming. You have three? What? Oh my god yeah. That's incredible! Three! Yeah, three

events, three events like you know you can do different strokes and length

distances and stuff. Just casually drop that. It's what's known as S6

swimming which is a classification in Paralympic sport. 200 meter individual meds,

sorry excuse me, 200 meter individual meds, 400 meter freestyle for short course.

That's great! And I feel like this also sets the basis kind of for like your life in

some form because you've had quite very interesting and which ones say like

a substantial life and also your perspectives are quite substantial. So if we go back a little bit

where did this all start? Like how did you end up being so interesting? Goddamn it!

So I think I would preface this by saying so I have osteoporosis and I was born with it as a

specific kind called Spondyloepfisile dysplasia which if you know the actor Warren Davis, I think

Warwick Davis, Warwick Davis. He's a short statured person he's about. He's quite short but he's been

in a number of really successful really famous films. He's a Hollywood actor. He's in I think

Harry Potter and a number of things like that. I actually don't follow this guy's career. It's

just somebody who has the same kind of osteoporosis I have. But what happens when you have something

like that is you it's the systems that exist around us will try to classify and read you in

very specific ways and often get it really wrong. You would call that in modern terminology like

ableism but I think it's deeper than that. I think there's an institutional desire to classify

people in different ways and it's universal and when you have something that is so out far of the

accepted norm then it becomes very very clear to see that. You have to fight against it for

your entire life. So this is where I think my sensitivity towards understanding

digital systems comes from because these are downstream. We use digital systems in order to

manage the way we communicate how we represent people and ourselves and how we store information

about ourselves and other people and that I think is intrinsically related to how we classify

people outside of that before the cybernetic digestion before we turn people from people and

attributes into representations. So downstream from that you mentioned the Paralympic swimming.

I guess the turning point the catalyst which we really drove that move into doing lots of

different things is I was trialing for the Paralympics as a teenager is about 19 years old

and this is after I'd achieved the world records and in the 200 meter individual medley

race that I was in the qualifiers for I did a turn the first turn from butterfly to backstroke

and as I pushed from the wall my hips disintegrated from the osteoporosis and I ended up

instead of going to the Paralympics I ended up getting a double hit replacement at 19 years old

and in the recovery from that it kind of wiped out my swimming career but it also made me realize

that I had to do like probably two things a little bit fatalistic one is that I was probably

living on borrowed time which luckily has turned out to not be true and the second is that that I

had to do a lot of different things and put like myself into lots of different places rather than

banking all on one one kind of thing so fast forward to 2015 I end up working accidentally on

signal in version one as part of the team that helped launch signal the secure messaging app

and get taken to Hawaii after volunteering on the github project for six months and then

flown to Hawaii and actually work on signal directly and I end up having a rather large

disagreement with Moxie Marlon's bike the the leader of open whisper systems and the

co-founder of signal over the use of phone numbers as an identifier which has become a huge issue

with signal which has been weird to kind of have been the quiet I won't even call it the source

because I never talked about it again I lost that argument obviously and that kind of really

blew me away I wrote an essay last year and was due to circumstances beyond my control only able

to publish it in March of this year but on our website newdesigncongress.org you'll find the

website they find an essay called who will remember us when the servers go dark which chronicles the

experience that I had that I would call if you like my origin story the second half of the origin story

in which I get a sense of the violence at the heart of the digital world and how it relates to

historically other forms of colonialist violence and imperialist violence

and not to say that this is not to say that Moxie Marlon's bike is an instigate of him that

but rather that like the technology lives downstream from these these goals and that it will it's

desire above all is scale and it will do that regardless of the cost and that was I believe when I

started to within I guess three years of that I would have moved to Berlin joined Tactical Tech

for a few years the data privacy non-profit and then by 2018 I would have formed new design

Congress which is the research organization that helps to understand the gap between what is said

to be happening and what is actually happening in digitized societies and then by 2026 now I've

got the research and development lab called Para Real Limited and so over the past nearly 10 years

now of this work we've been working very closely on understanding different ways in which

different I would say first principles if you like of digitization and how they relate to the real

world and those include digital identity and how we represent ourselves how we communicate with

each other over the network and what the the material costs are of all of this stuff and this is

the entanglement I think between those three things is what covers a lot of the kind of work that we do

and I feel like this example that you brought up or it's not an example it was very real and still

is very real with signal and the phone numbers is kind of a great case example to start from when

one's trying to understand the approach you take towards security and it's it's I was thinking

as your essay this is fine points out very acutely it's a missing perspective in a lot of projects

especially projects that aim to take privacy seriously or aim to be some sort of ethical or

utopian solution and so I'm thinking if we look more closely here because I most people are familiar

with maybe pen testing or testing for security vulnerabilities or using encryption you know but

like these very technically focused ways of ensuring security and I think this also ties into

the whole conversation of like different data systems trying to solve trust technically rather

than solving trust socially and but your perspective on security it shifts the lens quite strongly

and your essay kind of was a wake-up call for an entire movement you mentioned a little bit before

but like the history of like what what's been going on what's happening since that essay and

what's been what's what's emerged as like paradigm shifts one could say okay around about

July of 2020 so you know not to bring us back to that point but middle of lockdown where a couple

months into the pandemic a couple months into new design congress as a formal full-time

piece of work rather than a side project and I'd written previously my first I would say new

design congress piece called on weaponized design at tactical tech back in 2018 and your design

congress is your company if the research it's the research organization yeah as I mentioned a

little bit earlier so it's a research organization that we run that is building a body of work to

confront the gap between what is said to be happening and actually happening in digitized

societies so it's it's really what you're sort of taught what you described just then about like

the difference between um the belief that engaging only in device based or systems based security

brings safety and what is the actual outcome of it as like an example of the gap between

the two right that we that we look at very closely yeah so I wrote this essay in 2018

called on weaponized design which described how it was impossible to have a system it was possible

excuse me it was possible to have a system that could harm users without actually breaking in any

way and without really design it it systems or interfaces that harm users whilst performing

exactly as intended and these are the facebook news feed for example back in 2018 this is a

very quaint example now of being able to produce an emotional contagion in users through like

viral posts is an example of weaponized and the entire thing is operating precisely as is intended

to be but it can whether or not the designers are aware of it or not it can produce emotional

responses that are you know deeply harmful and we knew this as early as 2016-2017 when people

started doing research into the effects of the facebook news feed and its algorithms um from

that a number of other it's like one of the genesis pieces of new design congress and from

that there's a number of ideas that descend on that from that one of them is an essay I wrote

specifically to warn one of my beloved spaces even though i'm a little bit of an outsider

the pia to pia community so this i wrote this essay called in 2020 called this is fine optimism

and emergency in the pia to pia network in which i describe how the last 15 years had seen a surge

of interest in decentralized technology from blockchain projects through to that scuttle

butt activity pub and that after this period of time which kind of ended when bit torrent died out

and when the streaming services began like when Netflix began to take off i guess kind of the

development of the iphone and the move from desktop computers into smartphones which for years

couldn't really do decentralized tech very well um there's not you know what i could see was a

renaissance we could all see it at the time a renaissance of interest in decentralized technologies

that had explosive growth driven by the desire for platform commons and community self-determination

and what i say in the piece is that the goals that we had collectively back then are fundamentally

at odds with and a response to the incumbent platforms of social media cultural distribution

data storage and and so on and the piece was a warning that by the 2020s centralized power

and decentralized communities are on the verge of outright conflict for the control of digital

society and that the resilience of centralized networks and their political organizations

remained significantly underestimated by people in the peer-to-peer space and uh on the flip side

the decentralized networks and the communities that they serve have never been more vulnerable

um again this is written a few it was written a year before or actually no it's written at the

end of the first trump administration when a lot of people were beginning to feel a little bit more

relaxed that something better was coming after the first trump administration

and what i could see coming new space on the work they've been doing was like not that at all and

so my concern was that you could look historically and see what happened to decentralized communities

in previous decades you could then like apply that same thing because you know none of the issues

had been solved and as a result the peer-to-peer community was dangerously under-prepared for a

crisis-fueled future that has and i say in the essay very suddenly arrived at our door

and uh six years later that turned out to be true you know um and that's also like it's something

like that a lot of these uh older like now i do you scuttle about as an older network for these

precise reasons because they cannot hold up to actual security risks in real life and are not

therefore not scalable beyond other reasons of not being scalable so it's really taking a shift

and i think a lot of these newer protocols like peer-to-pumba have started embedding these

right this and also especially willow um has started embedding these thinking into their

methodologies and design of protocols which is also like stems back to this colonialist like

issues of who is actually building this technology right not necessarily always the people who are

at risk so this kind of thinking has started taking form in people's realization of building

these projects are you seeing that or is there still like a large discrepancy between security

in reality and security in theory that's a big question with lots of answers yeah

the starting the starting is that i hear you very clearly on the secure scuttle

about thing and i feel kind of the same way what was interesting is that when i released the piece

and i have had a some time to reflect on this and i know that part of this has to do with the

resistance to the ideas of the essay and part of it has to do with the antagonism that we have

at new design congress and it was a relatively new organization at that point um that piece

really divided people and i went from someone who had co-organized you know the first peer-to-peer

web spaces in Berlin like the very first one that um Lewis Center and um couple other people

put together at Trust back in like 2019-2018 um i was sort of going from being very participatory

in this sort of space to really making quite a few uh and i wouldn't say enemies but like people

who really didn't like what i had said and didn't want anything to do with me as a result of that

um and i named a number of projects this is something that i do because i'm not really i

think we would sort of run out of time but i named projects right i didn't say that people

were bad people but i named projects right and one of them of course was the secure scuttle

butt which i described as a what did i describe it as like it's a mercultry of forensics investigators

dream come true um where everything because everything every piece of um information

is permanently etched into a record whilst that is an absolutely elegant offline first system it also

uh given the fact as well that it was hosting a left-wing anti-capitalist political movement

was like precisely not what you wanted to have if anybody if anybody like in an

estate security service or something took the the threat of that seriously right it was a very very

it's unbelievable that nothing seriously bad happened during that time frame it was very scary

but but what i've noticed over the past especially the past three years so a year later i put out

with incense witch ray mccellvey benjamin roya christson day uh myself and peter we put together

and based off of an idea that i had about how we could start defending against these issues

um we put together identity primitive called back channel we started from this idea that

one of the threats in digital systems especially decentralized digital systems was that social

engineering attack was particularly effective where you could um pretend to be somebody on a

network and it didn't matter how much encryption or how well designed your system was um if someone

could convince you in the network in the peter pia network that they were someone else you know

equivalent of like catfishing someone on tinder but for purposes other than messing with people in

dating service something more you know it sounds side note yeah fun side note there we actually did

that on scottabut just as a joke because you know scottabots full of jokes it was more like a

humorous thing but like someone just created an account and it was four people and we had all

agreed to do this joke and it was just a joke um and we made an account called uh

dominic and dominic was the founder of scottabu and then everyone else started

verifying that dominic was dominic oh my god i didn't know the story

it was me kegs and a few others um and everyone else started verifying that dominic was dominic

but then dominic the real dominic was like no i'm actually i'm not gonna lose my account

and it was exactly what you're talking about here that like kind of co-opting someone's identity

in this case i find that like it's quite telling that we had to be five people who were friends

of dominic to agree to co-conspire against dominic but the the case stands that like the

digital identity of dominic who had made the software that we were creating a digital identity of him

on top of was having to fight us saying that we were not him i love that story it's so it's

lucky that he had those five rich friends and not like his worst enemies

it's yeah but like i guess the that's the thing that made it credible that we were actually his

friends because i guess if some random account came on and said this is dominic i guess it would

be because it's a social graph i i don't know i would love i would love your take on this like

because it's the social graph of trust verifying the identity does it become more or less safe

like in that sense i i don't think it's a case by case basis right and it depends upon

a number of factors and some of which are outside of the summer which are really temporal like

if dominic had been a way on holiday for example and away from his computer or you know sailing or

something or or just completely offline um it could be possible that someone who had been like

more closely tracking his movements with the goal of you know impersonating him for some reason

for a gain um could have used the time when he was offline to build a quicker version of trust by

like reappearing as a new account and then convincing certain people to sign that and then

trying to build that five person team that you're talking about but through convincing other people

that this attacker was him so you're right in the sense that like one of the web of trust is

is like what you've done is you've like speedrun the web of trust problem um where the five of you

because you were conspiring together and because you were friends with dominic that made it very

easy to do very quickly but you also not i i assume none of you were particularly skilled in like

actual adversarial like social engineering attacks like not that you have to be trained in it some

people are just naturally better at it than others as well as people who practice but

it's not like you know if you're doing it as a joke the stakes alert the the intent and the

motivation is a bit lower so that all plays in your favor the fact that you had the five people here but

it's i i would say it shows that it can be difficult it doesn't show that the five people

themselves protect you what it does show is the power of the social graph itself and the power of

of relationships that have existed outside of the network itself right that's where the real

power of social engineering exists and so gosh i wish i'd known that during the that period of

time i would have absolutely used that in the essay you would not believe god damn

no no no no no no no people would have done that for sure um but

i think we yeah i mean everyone has got about like we were utopian like trying to just strive

to build what we believed could be a better world and i think there's this yin and yang relation

between people who have your like gift of critiquing and and breaking things down to like the small

pieces and also seeing the holistic aspect at the same time i don't want to say penetrate but like

to to pierce the veil of like this utopia bubble but i think both are needed like we need to be able

to to that's another sidetrack which i would love actually to hear your thoughts on this but

just to finish the thought um like we need to have both the critical and the the visionary

together in order to actually have the forward momentum that's sustainable and i think a lot of

the visionaries in scuttle but uh want to not everyone because you got a lot of angry people

coming at you after this article but i think a lot of them want to actually understand how to

move forward uh sustainably and safely um but yeah so i think i agree with you i think the

difference where we would sit is that i i would call it aspirational uh in a sense because uh and

this is a i don't want to open a can of worms here but we could definitely talk about this

route is i actually kind of believe based on how humans have developed as a species um and again

with keeping in mind the wide experiences that individuals have and that cultural differences

between different um parts of our you know global society if you like not to use a again a really

seriously colonizing to a colonialist term the the human like humans are very sensitive to

scarcity it's one of the things i think that drives a lot of fear um i think that scarcity is one

of the things that's leveraged to uh weaponized populations into excluding and othering different

parts of society um and and so one of the tension points that i hold in my head is that utopianism

promises things that i think humans are for the most part somehow i want to say biologically unable

to accept but somehow intrinsically resistant to the and unable to really fully grapple with the

concept of of of abundance in like a serious way and i think you can see this very clearly in

examples such as like the modern the modern condition right where um in some ways there's

abundance like um for all of its deep flaws the number of people you know whenever you point

out the the like the crimes of capitalism you'll always have economists and other people defenders

come forward and say that capitalism has lifted the largest groups of people out of poverty now

sending aside that as an argument um uh that can be attacked from multiple different ways

one thing i would say to that is that i think it's interesting that even if you accept the very

like a very narrow definition of that claim or very narrow um interpretation of that claim that

there are fewer people in poverty in capitalist societies than they were historically before

they were industrialized even if you um interpret that as in its narrowest um version of that claim

um people we still have the society still struggles over the same issues of wealth accumulation

of fear of um resource fear you know cost of living things like that and without again

with keeping this very high level so i don't we don't veer off into another direct dimension like i do

worry that that utopianism as a as an aspirational goal creates a tension point between like humans

and that goal now sorry to be off topic that no i invited for the question i'm actually

personally really curious about this so i i've been thinking about asking you for a long time so

i'm glad it came up oh i appreciate that so i i but i'm not this is not nihilism speaking right what

i'm saying is that like i'm not at by any stretch of imagination analyst um what i'm trying to say is

that and this is where i do start to call on where i start to call on like some of the stuff that

happened after the essay um i believe you can be visionary or aspirational and somebody who has

a deep critique and the last five years has proven that um the the choices that certain people made

after the this is fine essay was published and after we completed the back general work for example

there are people who were inspired by that work or people who had parallel ideas that they committed

to and started building in that environment as that that world collectively opened up and that

dialogue started to happen properly and then there are people who didn't and what i find interesting

is that we tend to try to separate the visionary from the critic and i think if there was one thing

that i would say that was intrinsic to the initial blowback that i got from the essay and the warning

was that people the visionary for a number of reasons is very protective of the thing that

they're working on and i think that if you're a builder i it took to the detriment of of of of

that work to the to the detriment of all else like and and usually this this this manifests in the

sense where the the visionary will accept criticism on the terms in which they are developing their

this vision on so in in the case of decentralized systems this will be people accepting github issues

but not fundamental or philosophical critique of the thing that they're building in the first place

right which is but both of these are important right like the tech stack you choose is just as

important as who you choose to be excluded from the system that you're designing itself like

all of these ultimately are legitimate questions and you can't if you anoint yourself as the builder

if you want to inherit the arrogance of the builder of the vision visionary then you have to accept

the arrogance of the critic who believes that they can come and tell you why the thing that you're

building poses a threat and i would say especially not to invoke the kind of idea of ableism here

because i do keep this that side of me quite quite private um from expertise that can see it coming

right because it's expertise that's lived in that world in in a in a sense in exile from the norm

for you know my entire life in this case now of course like online much harder to tell i don't

identify with my osteoporosis online things like that but there were people who knew me i had people

who had seen me and and spoken to me in real life or seen talks i'd given and sort of you know there's

a sense of of of of like the how do i put this it makes me sound like i'm still really angry about

that this is fine piece and in some ways i think i might be a little bit um because i feel like

there's i'm sorry and that's okay yeah well i'm angry not because of the blurb i'm angry because

i feel like we missed there like there are ways in which things could be even better than they are

now in certain ways like the improvements the kind of consolidation around um decentralized

digital identities that are much more carefully designed than they were historically or the

the promotion of end-to-end encryption these sorts of things have been really powerful

but i feel like there was so much time lost in such a terrible lull it like in a lull in the

terribleness of the world that ah man if there was a way in which i could have written this better

or if there was anything i could have done to communicate the ideas better maybe that's not

that offering or just a part of a bigger system and what you did had a huge catalyzing effect on

larger pewter pew and local first ecosystem space and yeah to finalize that thought is that like

i believe that the visionary and the critic can be the same person and i think that to have someone

come at you and offer substantial criticism even if it's confrontational or antagonistic

if the criticism itself is deeply thought and something more than what you should do is x or y

or something that very clearly pushes a motivation that doesn't align or is an uneducated opinion

then i think you can kind of it's in i think it is also on you to meet that person where that

where they are in terms of that criticism and i feel like the people who are going to emerge

in the next couple of years as the leaders of decentralized movements because they they are and

will they will be and are being attacked right now um by the centralize um in common as i

described in the essay the people who emerge from that as the wind as the kind of systems that

are resilient to these sorts of attacks are the people i feel who have internalized their ability

as a visionary or as a sort of a leader of these kinds of systems and also someone who's internalized

the ability to critique it at an existential level um and that i think is like i think that

people who are unable to do that read really in danger um themselves their teams and communities

around them when you are completely unable to separate yourself from the vision and consider

the critique especially strong critique foundational critique as an indictment of your vision itself

what does one do when like because some of this foundational critique and i think what causes

some people and it just to sidetrack a little bit more because we have some time before we dive

into the next part which is also very juicy because a lot of people who are met with this

kind of critique especially when it's larger peer-to-peer projects that they've been working years on

and then they come to realize that there's like a foundational issue with the infrastructure and

how it's built and like solving these foundational critiques would mean basically starting over

from scratch like is it because because i think that's what kind of makes people

um bite down and i guess i guess what i'm leading on towards is that like in a conversation i had

a few years ago with Anyosha one of the approaches and one of the reasons that it was so difficult

for especially Skudel but to kind of swallow this critique is because we were built in such a

monolithic way or Skudel but was built in such a monolithic way that it was very difficult to

change small pieces here and there without changing the entire system and then Anyosha had

brought up this perspective that like in order to be more adaptable in order to be more resilient

in order to be more future-proof we need to change the way we build to become more modular

which i guess in turn also becomes an answer to the question i posed unintentionally so but

that we can respond to critique in a better way i'm just guessing because it's easier to change

yeah firstly shout out to the willow team sammy and um alio for being the urges of

recognizing that this is fine critique and kind of baking this entire philosophy into along with

cinnamon when they were um still with us um and when willow was called um earth star

that is a group of people descendant from secure Skudel but who like were even grappling it at

the time before the essay came out i think one of the first times we did a reading of the essay

itself was actually in their discord server in the earth star discord server which was like

only a few weeks after i published um you're right in the sense what what when you refer to

what alio she was saying in that conversation of of needing to produce a way of designing systems

it's way more modular i think that's 100 true i also want to say there's two other things to

and we can touch on this either now or in the future sometime because it's a big issue

that we have to change how things are funded for starters um part of the concern that there's

i talked about ego and fragility of the self as being a big part of the pushback to criticism

but you're also right that there are other reasons why people get feel really vulnerable why

team members who are building projects like this get really feel really vulnerable about

these kinds of critiques especially fundamental ones the first thing i'll say on that is that like

no matter what the nature of a digital system and the nature of compressing the variety of human

lived experience and the kind of material world around us into a digital system means that you

always make a trade-off and it's always going to be bad right there's always going to be a serious

existential drawback to what you're what you're building the question that you ask yourself

that you're as a designer as a like a protocol developer or platform developer as somebody who's

designing these systems is can i live with this and does this align with my politics completely

how closely does it get to how i position myself in the world and what i put forth as my politics

if you're a protocol designer if you're working in decentralized spaces and you're producing

you're engineering some kind of future you are essentially in one sense existing in this kind of

what i would call like a power real state where you are operating both in the digital world and

in the material world simultaneously in a kind of third space and you're creating that third space

so that others can be there as well right in a sense a little bit esoteric but it's really about

these creating these moments of charged sort of social interactivity that you that are mediated

through a ideally controlless decentralized system that's almost kind of like political writing

it's like writing interventions in the form of essays but your essays here occurred and rather

than reading them people are participating in them right and so and so beyond like beyond the

sensitivity towards that the two things i would say is one yeah absolutely what alia would

i talk with alia should pretty frequently i think that that's one of the core things that

they've impressed on me too and there's some stuff that we've been working on which we could

talk about sometime later um that really embodies this completely and then secondly that we have to

have funding we have to have a cultural change in the funding landscape on what the expectations

that funders have on decentralized systems because right now there is no margin for error

and error is seen as something that's has to be defended against or justified rather than

being seen as for what it is which is like an opportunity to to build upon

um and hopefully avoid you know systemic failures of the previous implementation

now of course i understand the risk like funders are this way as well because otherwise you end

up with people spinning their wheels reinventing things over and over again you know like reproducing

the same material every six weeks because they find a flaw in it and they have to start over like

the idea here is not to say this is why it preface this by saying that the questions that you ask

yourself are whether it's politically in alignment and whether you can sleep at night

um the idea here isn't to be perfect but rather that the the the work has to be

defensible not on terms of the mistakes that you've made or the or the the the blind spots that

have been coded into the into the work but rather whether or not it aligns with the politics of

yourself and to a lesser extent but still importantly the politics of the funder and then

building from that the other thing we desperately need are representatives in places like the

wc3 and other standards bodies who are um essentially pooling for the shed consensus on where we feel

these things should stand and what their standards should uphold and actually advocating for those

in the larger society because that's the only way as well that we're going to establish ourselves

for like longer-term strategies around funding around implementation around protecting ourselves

if we if we have that representation as well

and there's a whole world we're going to dive into which is like how does one get a consensus

in these spaces and how do we actually approach this and I think this also boils down a little bit

to this uh paradigm shift that kind of uh came about around 2020 and around the time of your

article um but I think one missing aspect in this conversation that we haven't quite touched about

or touched on which is we've been kind of talking around it but the methodology that you have

developed which mentioned it like three times everyone on the like listening is like what are

you talking about yeah and I guess that makes sense because it's often like the outcomes that are

relevant for people and but there's actually a process that you have charted out and kind of

created in order to discover these kind of fundamental challenges and safety concerns of

socio-technical systems yeah so you you have mentioned it to me and first time I saw it I was

what's axillic so it's an acronym there's two yeah anxiety and axillic yeah yeah yes there we go

and could you could you like help us like if if someone is coming into this and they want to start

understanding things from a social material security perspective where did they start okay

so to understand why something like this needs to exist we start first with what it's in response

to so maybe some people who are listening to this will well know this will be old news then but

there will be people here listening to this that don't know threat modeling is the idea of looking

at a potential looking at a set of conditions that you have right now in a digital system

and then trying to forecast in different ways focusing on different parts of that system like

the computer the network the inputs of like data things like that where an attacker might come in

and actively attack that um so things like um you know the idea of HTTP unencrypted

um web browser communication versus uh HTTPS uh the threat models around why you would implement

HTTPS is that the uh that the the data coming over the protocol in an HTTP request you know

such as your bank information or other data that you like might be sensitive that you've sent to

a website can be listened to when it's unencrypted by anybody between you and that server and so

that would be the threat modeling your threat modeling the idea of well if it's unencrypted

then it can be listened to and so out of that you know the simplistic like very simple example is

to then say well we'll exchange encrypted um keys before we then send an encrypted version

of the same requests over the network and the intermediaries can't read the data that you're

sending and receiving from this website it's like a basic example um go ahead can i jump in here

because an example that for me also when i was watching your talk at CCC this year um

you brought up an example that i was like kind of shocked by and it feels in hindsight like i

shouldn't have been but it also gave me a really nice perspective to understand how socio socio

technical threat modeling can also discover non-technical security challenges and you brought

up this case that was i think it was uh southeast Asia and uh there was a bank or some large large

collaboration yeah yeah in Hong Kong that was doing a transfer of large amounts of money and then

as far as i remember it correct correct me if i'm wrong um then there was this uh teller or not

teller but like a person working at the bank or company that was supposed to be transferring

the money and then there was deep fakes in like seven different people um who were representing

his co-workers right who were telling him to go through with this transfer right but it was a

false transfer and it became the highest of how much money uh let me i've got that number there

hang on um so to firstly to answer like to to like um you've got a really good memory that is

almost precisely what it was um the idea here is that it was a Hong Kong based firm 200 million

Hong Kong dollars um it's an uh it's a routine it's a it's a it's a routine phone call in a sense where

this is somebody who was able to who has the authority to transfer large sums of money inside

this large company um he's but the protocol is is that this person has to sit down with the CFO

and other senior team members and get the okay and do like a whole procedure where they talk to

each other and then verbally okay it and then they transfer and the idea here was that this

highest of 200 million Hong Kong dollars was completed because the people who were part of this

sort of security ritual which is meant to be you know face to face or over zoom call whatever

where you're verifying each other based on the presence of the other people um all of those

people involved except for the the target who transferred the money were deep faked um that was

in 2024 just over two years just over a year and a bit ago and um and so so getting to the

difference that that's a really good segue into the difference between sort of threat modeling and

socio-technical threat modeling the problem with threat modeling it's descendant from you know

darker department of defense style uh perspectives on seeing the world where you try to organize

the world into systems that you can understand and you eject anything that doesn't fit this model

that you created like the system that you've created cybernetic system as entropy as something

that um is an excess noise right as a something to be discarded from the system and as a result

of that all of the modern threat modeling practices tend to focus on devices on security systems on

platforms um on networks etc like the the actual digital side of things when in reality um the

consequence of that is that the social side of um the threats that emerge the kind of weaponized

design the the issues that I raise in this is fine and in the all the work they've done uh these

are of course intrinsically linked to the use of digital systems but they are fundamentally not

issues of digital systems they are what's known as socio-technical um uh consequences and social

technical issues around how we like the relationships that we have that are intermediated between

um digital devices and the custodianship of data the political downstream consequences

consequences of social ones etc and so what's emerging now which is very exciting is an entire

field which has been around for a while but is I think beginning to really crystallize is a discipline

within I would sort of say the humanities rather than comp sci around um viewing digital systems

in this lens and so for the last I would say seven years so from like 2015-20-2016

2016 was when I was starting to think openly about how to produce what I would call like a

generative framework that is a system in which you start with a handful of questions that then

emerge into your own taxonomy of of risk um but how do you build a system that is accessible that

you can use in lots of different contexts that de-center the system and instead refocus on

on individuals communities and and the social networks the political networks the economic networks

around these individuals um and still produce a working um understanding of the kinds of threats

that are individually experienced or collectively shared around the use or in at the insertion of

digital systems in these spaces and so I started this work I would say just after my clash with

signal and I had been picked up to become the um chief product designer a chief product officer

at a company called Spyderook which at that point was a very famous um end-to-end encrypted drop

box competitor that Edward Snowden had famously said in an interview was unable to be um uh

cracked by the NSA and that like gave them a wave of attention and and funding so it was the

head there of their of designing and helping the product team put things together with the CEO

and this is where this concept of anxiety came from an anxiety is a socio-technical threat analysis

framework that has seven vectors through which infrastructure projects can produce harm

whether through direct action architectural failure or external appropriation and anxiety is an

acronym or is a seven vectors you've got appropriation the capture of identity data or

infrastructure by a third party negligence unexpected governance failures specific to potential

decisions that a designer may reasonably make this is you know everybody from a protocol designer

to a um an app designer exclusion failures to account for material conditions that temporarily

or permanently block access for individuals communities or entire populations out of the

system or downstream from that other services that rely on that system in personation which is a

social engineering attack vector attacks against infrastructure staff or users or that use the

infrastructure itself to impersonate an attacker exploitation which is the abuse and attacks driven

by system based incentives so these are things like um uh where you have like when you add a

a token to a decentralized network and then you get like fraudsters come along this is like

exploitation where the goal is to like the attacks are accelerated as a result of the financial

incentive toxicity direct harms to the social fabric made possible by the project and yielding

attacks that rely on coerced consent anywhere in the infrastructure so anxiety takes these seven

vectors and um through a series of um pedagogical sort of designed and we could this is another

talk for another day there's a whole set of pedagogy um inspirations that I pulled from as part of

this there's different methodologies within that to kind of tease out in in a participatory way

some of the threats that emerge and how they how they interact with each other and then once you

have an understanding with that you can do this as a practitioner or you can like bring communities

into it and work it sort of flexible enough to do both the kind of response to anxiety is a

design framework that we call axilic like exile and axilic um is is a much more looser system because

it doesn't allow you to um it's not like a prescriptive thing like human-centered design but instead

it has nine core tenets so built for post-cope this is the worst case as the worst case scenario

is your central constraint it's an empathetic interface design that prioritizes cognitive

diversity and intellectual sovereignty relationship-based identity a person is a person through other

people collective access to emerging tech this is things like economic justice and resisting

cloud coercion self-hosted infrastructure by default and it uses um you know video game inspired

accessibility so things like how kids can set up Minecraft servers by themselves and bring their

friends in without having a third party involved um no network by default connectivity is seen as

a liability and it requires explicit permission deletion is fundamental right decentralization

is worthless without it even though deletion is one of the hardest things to do in decentralization

break these are systems that break the frame these are inter-processed integrations anti-siloing

of data and connectivity and um uh designs that exist beyond the electron app window things that

can move through a system and kind of the last tenet would be the end of what we would call what

Fukuyama called the end of history the end of the end of history where we consider permanent

instability as the operating assumption rather than the belief that liberalism had triumphed and the

world was a stable place consequently and that the kind of values that axillic ab once again is

an acronym are that an axillic desert an axillic system is one that is ephemeral with graceful

graceful degradation that allows for decay exits with allows for exit reversibility right to

deletion it is intentional with explicit adversarial modeling and non-weaponizable design it is local

is um speeches infrastructure independence human rebuildable systems and pluralism economic

independence informed as in participants understand and have cognitive agency and consensual in the

sense that participants express their own sovereignty and it allows for explicit permission it demands

explicit permission for participation in the system um and what I would call it based on the

work that we were talking with the more broader work that I do is that it's the act of knowingly

building within the the power reel again this idea there's a system designer you're kind of

building between the real world and the digital and that third space is where a lot of I guess

the ideological struggle plays out today these are the things that we use that have been the

bill in in in the in the background I guess since 2016 um to you know over the last year and a half

I've been more formal like formalizing more directly I'm hoping to actually publish this next week

like the first draft white paper for this stuff I hate using the term white paper the first draft

text if you like all these two systems um yeah I'm really looking for a threat because that was

actually my absolutely next question which was how to learn more about this and where to find it

I have the URL in my head and the URL is um newdesigncongress.org slash en slash pub slash anxiety

dash exilic that will be the URL awesome and I will link it in the notes to this episode yeah and

if that 404 is on you then you just go to new designcongress.org and it'll be on the front page

super duper so I'm thinking right we already before we even started interviewing realized that

there was going to be so much part of this like conversation that we could have that could be had

and that we would like to fit in that we kind of honestly knew yeah exactly we already knew that

we were not gonna fit at all in one episode so I'm thinking brainstorming here uh one potential

next episode which I would be really curious and I would love to experience is tackling this

challenge of digital identity but specifically distributed or decentralized identity absolutely

in uh from the perspective of these frameworks what do you think well we've just there's a whole

story there's a whole backstory for this too but we have been sitting on a report that we weren't

able to publish um on digital identity until very recently so that would be a banger of a starting

point I think um there's a lot as a sneak peek I would say the way this would we would be talking

about a four and a half year long uh study that we conducted on the failures of digital identity

plus the parallel track that was emerging uh both of as a set of the learned experiences of certain

protocol design as in developers combined with the this is fine essay and some of the work that

new design congress did in 2021 with incand switch which either directly inspired or validated for

others um the idea of a different kind of identity primitive and sort of five years later four

and a half years later seeing some of the work that has either directly or in parallel emerged

as a result of that there's an entire collection of digital identity systems distributed identity

systems uh that really I think offer an example of this axillic um methodology that can take into

account some of the issues that exist today that historically hadn't been considered important

we could do in a whole episode on that very easily here amazing I'm already excited in sitting here

jumping in my chair but with that said we will have to leave the rest of that conversation

and keep the listeners on a cliffhanger because this is the wrap for today's episode and

greatly looking for us to the next conversation we will have Kate and is there any last notes

where do we find you how do we connect anything any shout outs you'd like to make oh to the team

new design congress uh this is like Lewis Center and Benjamin Royer and all the people who have

worked with us in the past my lawyers that would be a good one that's for tomorrow's next episode as

well and um yeah I mean I should do the shout out to allie Rish for that like the teams that have

been working on stuff uh over the last few years um and I guess if you wanted to find me

I am at post.lurk.org.com at Sheba Computer S-H-I-B-A computer uh and then yeah you can find our

stuff on newdesigncongress.org and finally if you would like to do some pretty heavy duty and very

very helpful um socio-technical threat modeling uh and analysis of your work come talk to me uh

we can apply some of the anxieties framework stuff with you and get some really really good

actionable outcomes as a result of that kind of work so if you're working on something and you want

to um and you want to really analyze it about how it will work in the world let's talk awesome

thank you so much for joining Kate thank you so much for having me I super appreciate it this

has been really fun yes I think it's the same and I don't have a wonderful rest of your day

with your grumpy little Sheba's right now and yeah I go out yeah they're okay yeah as long as

the rain stops we'll have a great one thank you bye

Joe