Tools For Thinking
John Borthwick put out a call for Tools for Thinking:
People want better tools for thinking — ones that take the mass of notes that you have and organize them, that help extend your second brain into a knowledge or interest graph and that enable open sharing and ownership of the “knowledge blobs” you create. Entrepreneurs are cracking open the category and building these new tools.
I wrote about how this made me think of the Memex
I'm going to take notes on the conference here.
I'll be taking notes on twitter, and posting them here afterwards https://www.kevinmarks.com/toolsforthinking.html
video starts here in 3 minutes or so https://www.betaworks.com/event/render-tools-for-thinking
Tools for Thinking help us ideate, think and work better. You'll see people using tools, and we'll also be using plexus to help share thoughts
Neel Baronia: We're running a ThinkCamp accelerator program at betaworks.com/camp
Neel Baronia: we'll be organising some follow on talks over the next few weeks too
Part of the purpose of this is to kick off a conversation. Jerry and I are going to start with a couple of rants about this.
we have access to all this information, but we don't have access to great tools to make meaning out of , which is why I think we need better tootls for thinking.. We are forming our minds to be mass consumers and not great thinkers
The moment we are in is also important. We have seen a lot of development of graph databases and machine learning, and with web3 we could have data owned by everyone and shared across apps
We're starting to see more semantic spatial work begin, and I think we're going to see UI to make this work better
We need to think hard about how different people think. I first thought of tools as memorisation tools, but I never went back to them. I went back to pen and paper because the associative function was more important than the memoty
I find that going for a walk and going to sleep will often find a new association for me. Let me hand over to Jerry, who has two brains
I have a piece of software called 'The Brain' that is a mindmap of my thoughts - which is software from a tiny company in Los Angeles
this is the same file I opened in december 1997, and there is a half million of these 'thoughts' in the mindmap - I'm like the guy who made the palace of fine arts out of toothpicks
Raise your hand if you thought that HyperCard could have been the web, but Apple was stupid
Most people use mappy tools for mindmaps etc - there are lots of these - what i want to show you is someone using one of these tools for himself
How can we make a social memory so everyone can use their favourite tool, but connect between them? [km webmention!]
By putting things into this tool, I have improved my recall because by adding 'thoughts' to this brain, which helps remind me of connections between them
how do we connect these tools? our datasets are all siloed, we need a bonfire in the middle to share around
If we do this right we can help make better decisions as a society, and we are in 5 simultaneous crises right now
The web is stuck in Web 1.0 language- ideas go into books or PDFs, so we can't connect them well
It's a big space, and there's a lot happenign in this space, we're looking at the history of computing and the way it's moving forwards
PDFs are where information goes to die
Dan and I founded Readwise over a passion for reading and spaced repetition. It's a tool to help you read better and retain more of what you read
we want software to improve reading the way it improves writing - people were highlighting things, but never revisiting them, so we showed them back to them
we then got asked to bring the notes inot other tools, so we're like the Zapier of annotation
I thought this was a memoisation tool, but when you started moving highlights to other apps it made more sense to me
When we staretd Evernote was the main note taking app, over the past 5 years we have paradigm shifts over time - vernote made it 'save and search'
the next shift was Notion 2015-2019 - block based architecture - pages, images are all block that you can nest infinitely, and also multiplayer
We think tbe most recent innovation was Room - the first graph database that broke through with backlinks and transclusions in 2021
There are a lot of spatial and visual writing tools going on, but we're waiting for the next inflection
why did Evernote get stuck?
Evernote nailed the consumer usecase but never got the enterprise or multiplayer part right compared to notion- they got distracted during their growth by recipes etc
How do you think about the funding models? Evernote and Notion raised a lot of money, but there are people looking at other models
The promise of the internet is to be able to start a business - it is now easy for one hacker to build a first version before looking for funding
We tried for 2 years without raising any money, so we charged a monthly fee and walked up the SaaS ramp until we could support a team
Enterprise SaaS is the most proven busisness model, when you're in the consumer zone like readwise, it's not clear that this is venture scale
The SaaS economics of Web 2 tend to silo data and system in a specific way - Gordon Brander is going to talk about that
People will put a book into Readwise as paragraphs and review it via themed review, then put it into their note taking app and sequence it. Theer is an mazing writing product waiting to be made there
I have tried the GPT and AI writing tools, and I coudl write better myself
the person who writes and the person who edits can both be you if you use this to separate yourself
AI is not the product, but there are opportunities for it to help enhance your work
Starting with a blank page is broken - you come to the page with all the ideas but they are in different silos - Machine learning tends to overpromt, and you get fascinated by the machin instead of thinking
it's very common in our space to get stuck on the system, so they're optimising the process rather than writing or thinking
it feels like this space has become more mainstream, but it is till really early adopter - we need to find ways to use it to improve things in our lives
What about Audio? We have seen the rise of podcasting and there is lots of audio, but it is challneging to move inot this space as you need to transcribe and excerpt
we've been blown away by the growth of audio- books are growing 1% a year, audiobooks are growing 15% a year
we have a text to speech option and it is being very popular. Transforming audio into a block or a highlight - there are tools that can snip, but it needs improving
we're in the final bit of the uncanny valley for text to speech and speech to text, but ti is getting there and it will be free and everywhere
the audiobook side is dominated by one company at 95% and is a very closed format - we're seeing innovation on the podcast side and TTS
Silos all the way up and down - saying 'siri take an airquote' gets you stuck in OS level and amazon and audiobook trapping
we went on an adventure several years ago at Betaworks with an annotation app called Findings that Amazon killed
how have you seen the usage of your product change over the years?
we started with spaced repetition - the daily habit hook - we got pulled into the note taking apps, especially Roam
the explosion fo different apps has been incredible to watch - it used to all be Evernote, and now there are so many requests for integration
Have you thought about Enterprise, and what does this look like there?
we talk about single player versus multiplayer, but we haven't found good ways to do multiplayer reading
our main interest is in helping people read better, and we'd love to jump on enterprise Saas, but there isn't n analog compared to Notion
You talked about information density - you can read twice as fast as you can listen - are people reading slower?
it's a divided attention thing - people are listening while doing other things so there is less information going in. People who listen and read tat the same time seem to absorb things faster
people are saying there is no attention span left - people are Wladen Ponding by going to cabin and reading paper insetad - we want to fight tech with tech
immersion reading - parallel audio and reading - cna help you get this
you're now unexpectedly owning the plumbing between the thought products- how do you make it better?
we backed into this owning the plumbing sitiuation - it would be great if there was a standard format for annotation- there are some ways to do this but it;s very web focused
we have annotations for PDF, kind, podcasts audiobooks - there's no spec flexible enough for that - if you cna please help
you're talking about annotations and highlights - knowledge blocks are important - I give both Roam and Twitter credit for that context too
we have our own reader app, and the promise of the block drew us in where every paragraph was a block
we found that 'blocks are too soupy' - they're very fluid - there is a phase change from the fluid blobs to a coherent text - blocks to docs
the challenge is to take the doc and make it into fluid blocks, and connect them back together again
highlights and notes are the connections between the media
with a 100 twete thread, our users want them as a single document that they can highlight as a document
I love readwise as a way to jump from tool to tool - could we customise the ingestion somehow? Also readwise for slack
the formatting came out of Roam - we let you take your notes and sync it out; before Roam we just dumped highlights - Roam users wanted more choices
after a bit of that we gave them jinja2 templating language to export info in a flexible way
I have an alpha for readwise for Discord I cna show you later
have you looked at using this in education? Would students be forced to use it
we have never had any success with academic or school users, we tend to appeal to autodidacts instead
Subconscious is social notetaking - I want to talk about something a bit bigger - the internet is already a tool. for thought - it should get good at it
the internet has not yet produced tools that help us cope and make sense together
when societies first became complex we had a sense making crisis, we have another one now with the climate crisis
we need to get off the collapse trajectory and onto the sustainability trajectory or the go to space one
we have all these great tools for thought, but apps trap thoughts in SaaS silos - the same origin security model causes this
each app lives in it's own pocket universe by default, and you often need bizdev to connect them which is no way to scale
we need a protocol for thought - what if you could draw knowledge form anywhere and connect them together, like Xanadu
we're building Noosphere, a world-wide knowledge graph - permissionless, decentralized over IPFS, user owned and multiplayer, belonging in to everyone
It's like HTTP - belongs to everyone [so use HTTP - km]
you get cross-app transclusions and back links, change history, and user owned back pack, and Lots of Copies Keeps Stuff Safe
libp2p from IPFS - we have p2p petnames, versioning like lightweight git, and signing with a UCAN key
we have a generic envelope called memos that has metadata, body ContentID, and a pointer to previous (parent) ContentID
UCAN uses and OCAP model to enable access and editing - you own your keys [so you have to keep them safe?]
the Sphere server talks to IPFS, maps this to your public key and paths to ContentIDs, then it gossips them so you have lots fo copies, and it bridges to the web
running on the device was too battery intensive, so we run it on a web2 server [so you don't replace HTTP after all then?]
we have a discord at discord.com/invite/wyHPzGraBh
We haven't drawn a boundary around the 'tools for thinking' idea - note taking isn't just it, but adding links and backlinks matters too
We now have lots of people on YouTube who explain their methods, but it is all "do exactly what I do" "if you're not using colours you're not mindmapping"
throughout this history we see that people who use certain tools find things to do with them that people who creaetd them didn't have in mind
the internet is a good example - there is a myth that it was a bombproof communications medium - the ARPA researchers under Licklider we're scattered, adn they wanted to use data from one to be processed ona. remote one
afterwards they used it to communicate with each other about science fiction instead, and Licklider and Bob Taylor recognised that wahat was emerging was a new communications medium, and wrote a paper "the computera as a communications device"
I think about StackExchange - how did programmers work before this existed - we need. a StackExchange for the Tools For Thought community
the lecturer isn't as useful as a group of people learning together - colearning
how can we do more useful crap detection?
Crap detection has become a Wicked Problem - when my daughter stared using search engines 20 years ago, I warned her that books had editors, but the web didn't so you had to do it yourself
now the arms race between the critical ability to sift the good information from bad versus the ability to target bad information or misinformation to you is not going well
we need to help people search for medical information and not kill yourself, and we have politcal misinformation that is targetted at people
we need a combination of social and algorithmic filtering - I would curate my own list of trusted experts to filter materials for me
teher is a thing called sift - to check on the veracity of. a page, get off that webpage, search for the authors name and see what others think of them
I remember on The WELL, where I met you, that there was an experts forum that would answer questions usefully
what about the problem fo the human mind looking for self-reinforcing loops, linked into the net?
Confirmation bias is a problme - we're more attuned to evidence that supports what we already beleive, and emiotional attachment to existing belief
James Fishkin has been researching Deliberative Polling - he gathers a diverse group and poll them, and then have them talk to groups of experts about it and ask questions, and poll again and they move to the factual
there are some powerful human prejudices that keep us from learning things, but we can find ways to work around them
the group is all-important - we're not disembodied brains thinking alone. Experts on the WELL still exists - I was on it this morning. One of the fisrt things I was excited about was when someone asked a question, I enjoyed answering
I asked Mark Smith why people would give answers to strangers and he said Social capital, knowledge capital and communion. It's nor just tools for individuals but for groups
is there a place for authority figures in the future without going back t gatekeepping?
I talk about the authority of the text - how much can you trust the information from a text or a person? The college professor used to dictate the truth that you transcribe
I found students were more engaged if they were given an question and had to find it for themselves - authority is confrimed by other people; now you need to think about thw community that is doing the confirming
I read a letter by Rothbard against Polanyi criticising him for his methodolgy, while violating all the thinsg he was accusing him of
our brains operate on emotion as well as reason - how do we use emotion in these second brains?
understanding some of our emotional biases is very important - the book The Media Equation is worth reading - they did psychological tests, but swapped in a computer or a cartoon instaed of a human, and people treat them as humans
we haven't made the distinction between a human and a synthetic human. There's a problem with teaching critical thinking, questioning parents, teachers and politcians is not popular wiht them
your example fo using analog mehtods with paper to write your book - spatiality and tactility matter there
there is someone who puts tactility into the computer, where moving objects change the computing environment -
we can do 2d with a mouse, but we haven't managed it in 3D and VR yet
The book The Extended Mind - there are people who are and aren't aware of their heartbeat - stockbrokers who were aware of it made better decisions
whats the most psychoactive experience you've had around a computer?
I gto access to the Alto at Xerox park and was able to move text around with it while writing, and that convinced me it was a better toolo for thinking
Englebart said that automating the lower level thinsg with a computer frees your mind up to think about higher level things, and we do see that now
after a 30 year hiatus I did take LSD again a few years ago and put on Google Earth and realsied that Google was building an AI
Alan Kay talked about building models and simulations, not just pure text - what tools do we have for that?
there's a strogn connection between our intellect and our ability to manipulate space - visualising rotating 3D objects. I haven't seen what VR can uniquely do, but if you give attributes to objects in space that could be a way to see relationships
a lot fo the VR thinsg were done in the 1980s already, and we don't see much new - Zuckerberg's metaverse looks a lot like Second Life to me
what si it going to take for the things we are building here to get out to people out there?
I built a notes tool in 2018 that helped me make connections between scattered pages, and my friends thought the tool was trash - they didn't want to use special grammar to take notes
I made. seocond tool that automatically made connections without needing markup, but my friends didn't see the need for the tool
The connections they were looking for were not connections between their notes
at plexus now we're building a new kind of online community - a shared brain - go to https://render.plexus.earth/p/render
if you type a thought inot the box, it will give you the thought that matches it most closely
at re:collect we're building an ai powered creative assistant
I was a cognitive neuroscientist in academia, and we're using machine learning to augment your ideas
we have unlimited storage space on our machines, but we can't reason about what we have in there
what you want to store is hard too - we are overwhelmed and we soothe that with bookmarks and notes, but we have to remember to look for them again
we have to many breadcrunbs - too many tags, folders etc. We sidestep that, by storing everything and making connections
my past self was a jerk - I don't remeber the keywords or tags I used before
we bring in articles and tweets that you are reading, and let you write things in the tool - on the right hand page you get ideas that are related by the machine learning models
I can highlight any sentence in a document or on the web and we will show you related ideas from your store
you can't operate on words like you can work on numbers, you need to rewrite rather than quantify
just as we can manipulate colours by converting them to RGB values, what fi we can convert word meanings to a space that we can manipulate
by putting 3 different sentences in space we can construct sentence that are related but inbetween
by using language embeddings for eg like vs dislike, we can warp how feelings affect the sentence
we want to encapsulate ideas and be able to do algebra on ideas
we have all these different reading apps that feed into readwise for note taking
we realised about 6 months into Readwise that we were actually building a web browser
my hypothesis is that the capabilities of machine learning is why this area is getting exciting again
we now have the large language models and transfer learning, which means that we can take advantage of these larger models
we also have the ability for people without an ML PhD be able to use the tools to o intersting things
the new thing with models like GPT3 is that they are extremely laerg- internet scale datasets - they imporve at a linear range wiht amount fo data
with transfer learning you cna have a pretrained model that understands language generally, and apply it to your own notes
what are the areas that you cna use this for?
a lot of people are intersted in Prompts at the moment, but they are a skill you need to master, and they have discontinuous problmes, changing a few words can go astray
beyond prompting, you can start to build interface abstractions that assume the computer understands text instead
I'm not sure that computers can work wiht meaning - I see the lnaguage models as one componet of different hings that we cna bring together -how much they understand is not quite liek we do
I cna say 'make a computer program that can mimic my memory' but it will need a lot more human in the loop to get a sense of what the language model is actually doing
we cna study how humans form concepts, but we can study that, and use different methods to draw that out
is the gap of understanding going to be fixed by improving the models?
we're not going to get to AGI with just adding parameters, but before that we can use multiple kinds of models - language + vision+ higher order control to combine them
what limitations we have now will change in the next few years?
one fo the limitations we have now is that we need labelled data for the task that you are doing - unsupervised learning is not there yet, though it is being worked on
the other issue is interperetability - you look at it and go 'whyyy did it do that?' - it is a limitaiton but it will change over time
you cna grab GPT3 off the shelf, but ti will make a lot of errors if you don't understand how it was trained, they won'r be substitutible
controllability is like interpretability - how can I steer the model towards what I want? I want it more factual or casual - we don't have that kind of smooth continuity
so we don't know how to guide the model to not be racist - we see disclaimers saying 'we don't know how to control this model to not be offensive'
I spent a lto of time advising companies on advising how to build ML prodcucts - if you;re a small company you don't have a lot of data available, so cna you find models that you can run on reasonable infrastructure
for every bit of data transformation and pipelining you need to design it to be computationally feasible - if you area alarge company you may have more access to processing, but sclae still matters
it is how you break down the problem and find an appropriate model for each piece of it
the model can not be your product - that is going to be easily replicated by others; interfaces and better data collection is how you can compete
The demo I was running earlier was pinning the GPU for a single user so it would not work as service yet
retraining in real time is going to cost a lot fo money, or retraining will have time lags that makes it unfeasible
if there is a specifi thing you need to do you can make a smaller model to replace the big complex one
don't use models if you don't need to - you cna burn a lot of CPU doing something that could be simpler
it's a really tempting thing to do as a founder is to bet on Moore's law to catch up wiht your complexity, but the models are growing too
text completion is where we started out, where do we go next in interaction models?
one space that people aren't thinking about is where the features are invisible - put the fact that the models understand language they can do search better
there are lots fo ways to make software features better by adding naturla langauge control
a lot of products are changing the product in response to user interaction that are invisible
if we can make a change on what we actually now form your knowledge rather than general sentence prediction
we aren't used to being able to speak to things that aren't a human, so making it clear that you aren't talking to a human is important
we're also thinking about taking something that ahs your notes and writings and what you read and make it more like reading you than the text of the internet
reading is tiring -there might be better ways to expose the output rather than just giving them huge walls of text
the limiting factor is not always the ML but the interface design
our brains are good at navigating 2 or 3d spaces - can we turnt he structure of an essay into a spatial navigation task
one thing I tried was getting a summarization algorithm to highlight different parts of the essay depending on how important it thought it was
I want to start talking about new kinds fo interfaces like this that can take advantage of this kind of processing
it's easier to see the gist of information from a 2d or 3d space- if we cna bring that back it can be easier to make sense of large amounts of information
can I get a visual or higher level abstarction of different sorts of information? Or can I highlight bits to feed back to the machine
where can we find unintended negative consequences?
when humans work with systems that talk back, we implicitly treat them as an authority and we often trust systems naively - we need to be careful not to make the information too emphatic
on the modelling side there is a lot that goes into AI ethics - if you are using system that can change on its own, you need to think about the disaster scenarios and what about thraiing data that goes in
we really need to have a good sense of the base models that we are building on top of - what is the most evil thing that it could do?
we have a few large organisations making bigger and bigger models that the rest of us use to build products, and any flaws in those few models can influence everyone else
prompts are very input/output based at the moment - could we change how we use our tools to give more info about meaning?
I am wondering how we give more information as an input - understanding changes of voice emphasis or hand pressuer to influence results
there is no understanding of concepts in GPT3 - you can see discontinuities in the outputs; the human model of concepts is not in there
text is very self similar- a lot of text looks the same - could we use a large language model that adds a symbolic representaiotn of the found structure?
with the large models there isn't that kind of structure, but we could maybe abstarct repetition into tables
could you replace the text with a richer representation
if we're building tools for thought how do make sure that we are getting smarter, and not making smarter tools that make us dumber?
this is like GPS making us worth at navigating - we cna offload information or knowledge to others or to machines- within a group different people remember different kinds of thingsd
there is a danger that we over-rely on tools, but we are already taxed with what we need to remember so off loading some is ag ood idea
I remember getting off the school bus at 8 and having an ocular migraine - i did not lose binocular vision, but I could no longer see a 3D world - I always remember that as a way our brins construct a 3D world
you can search about topics, but can you search about grammatical structures and which way verbs operate
time, sequence and dependencies are structures that we can model, but don't naturally occur in language models
explictly modelling stucture and what are support beams, and what is filler is differnt form modelling shape and surface image
in theory law is structured, coherent and defines things, but if you look at enough of it you find contradictions and loopholes
have you seen lexon? it's an attempt to create a legal language for smart contracts that can be human readable too
as you get older you begin to forget things - the polite explanation is that as your brain gets full it takes longer to search
David Allen, the Getting Things Done guy, says that we have lots of open loops in our heads, so we should get them out into a reliable system that brings them back to us
I avoid using people's names, because when I was growing up it was really bad if I called my mom by my stepmom's name, so I don't learn names as I won't use them
how do you build a real shared understanding with people over time?
I went to a misinformation conference, and there was a professional misinfo guy, and he explained that he didn't care about info, but it was a good way to make money, so look for that behaviour