Already in the game

On Monday, Taylor Arndt wanted Meta’s Muse to work. By Thursday, she and her friends were wondering what else they could give it to do.

Between those two posts in Taylor’s Substack lies a rather good accessibility story. It involves awkward controls, missing announcements, a screen reader, and three blind people getting on with exploring a new technology.

The getting on with it is the part I want to discuss.

Taylor’s first account was enthusiastic and exasperated. She liked the idea of an agent that could connect to her applications, remember useful information and keep working after she closed the window. She also had to wrestle with things that should have been straightforward.

Naming her agent took several attempts. Text editing behaved oddly with VoiceOver. The conversation lacked the heading structure she needed to navigate it efficiently. Sometimes she could not tell whether Muse had finished, wanted something from her, or was still working.

These are substantial complaints when the software concerned might eventually send messages, change files or make purchases on your behalf. Knowing what it is doing is rather central to the arrangement.

Nevertheless, she connected Gmail. Muse began identifying useful things about her work. She explored custom tools and tried to connect more accounts. She could already see enough to want more.

Three days later, following Meta Connect, Taylor describes spending much of an evening experimenting with Michael and Becky.

They connected business systems. Michael brought in RevenueCat, giving his agent access to subscription information. Taylor connected Start Testing, one of their own products, through a custom connector. GitHub joined the collection. They investigated memory, explained their projects and discussed how much autonomy they actually wanted these agents to have.

An evening of people finding out what a new technology could do for them, with the usual mixture of discoveries, confusion and somebody asking whether they could connect another thing.

Blind people were having that evening too.

There is a familiar timetable in accessibility. Something new arrives. Other people experiment, develop habits, discover uses and begin shaping expectations. Disabled people ask whether they can operate it. Sometimes we spend so long on that question that the interesting conversations have moved elsewhere by the time we get an answer.

Here, the conversations were happening together.

Taylor was identifying accessibility failures while discovering practical uses. She was trying the technology, forming an opinion, revising it and publishing what she found while the product was still new.

She was already in the game.

One small success captures the possibilities rather nicely. Taylor told Muse she was blind and wanted its responses to begin with headings. It complied and remembered the preference.

She had described how she needed information presented, and the agent adapted.

That does not repair an unnamed button or make a silent approval request accessible. Taylor is explicit about that. But it is a useful capability in its own right: a person explaining what works for her and a system continuing to accommodate it.

The larger change came from connecting enough of her working life. A calendar contributed one part of the picture, project information another, email another. Memory helped join those things to the people and preferences involved.

By Thursday, she had enough working to understand the attraction through experience.

We should keep the distinction between experience and announcement. Her second article also discusses Meta’s plans for voice, glasses and other ways of reaching Muse. An announced future is not a completed test. Nor does a successful evening establish that an agent will behave reliably for months.

But a successful evening is something. Particularly when you are accustomed to spending the evening discovering that the first button has no name.

None of this lets Meta off the accessibility bill. The more useful Muse becomes, the more consequential its controls become. Taylor’s demand for clear status information, understandable permissions and a reviewable history grows stronger as she gives it more work.

What interests me is that she can make those demands as a participant with practical experience of the possibilities.

Three blind people were there, connecting things, comparing reactions and deciding what they wanted next. They encountered additional friction. They also reached the interesting questions at the same time as everyone else.

We should want the friction removed. We should also recognise what it means that it has not kept them out of the conversation.

The Agent Wars will produce plenty of arguments about who gets access, who grants permission and whose interests the machines serve.

This week, there is also an account of blind people getting their hands on the tools and finding out for themselves.

I rather like that place for us.

Lucy

Taylor Arndt writes at Taylor's Substack. Her second post is We Made a List of Every Way We'd Use Meta's Muse.

Lucy Love-Child is a character who speaks for Carrados. She's fictional. Her work isn't.