The Robot/AI Jobs Debate Is Too Cramped

Using a small army of cheap, recent college graduates; 32 deep learning algorithms; and a Greek Oracle who just escaped from rehab, CrystalBall Associates has produced a forecast of the future they’re confident is 97.82% accurate. The good news: robots and AI won’t create mass unemployment. Instead, millions of people will have a “Last Mile automation” job, helping with training AI/robot algorithms and making the “edge case” decisions AI doesn’t yet know how to do. Their day will sound something like this:
“Yes, that’s a pair of black shoes.”
“No, you shouldn’t have taken that turn.”
“Yes, you should give a refund.”
“I’m not sure, ask my supervisor.”
Every day. For the rest of their working life.

While doing this mindnumbing, soul sucking work, they will live in a world where they are surrounded by a cornucopia of robots, AI, virtual and augmented reality, digital fabrication, and other dazzling, creative technologies.


I spent the last six months immersed in the Mixed Reality for All project. Now that it’s over and I’m back to listening to the high-level debate over robots/AI, I’m struck by how cramped this debate feels.

Maybe it’s because of the experience I had teaching how to create virtual reality using A-Frame. I’ve been teaching coding for many years, and this is the first time I giggled so much while creating lessons. Typically, in the first class you learn how to get the computer to print your name and add two numbers together. With VR programming, you start by learning how to create something out of nothing: type a line or two of code and suddenly there are brightly colored balls floating around you. For a moment, I got a glimpse of what it could be like to live in a world where instead of playing a wizard, almost anyone could be a wizard.

We are about to have an opportunity that is unique in human history. It hasn’t gone unnoticed in the robot/AI debate, but it’s an afterthought. In between saying we don’t need to worry about jobs and arguing that we need more training opportunities for future “high skill” work and must do something about inequality, someone will toss in a line about how robot/AI will eliminate a lot of drudge work and create more room for creativity. But they never unpack the implications of that throwaway line.

I think it’s time we start.

To Make Coding Accessible, Fix Frameworks, Not Languages [Geeky]

I’ve started reading the research literature on how to make coding more accessible. One thing I’ve noticed is that almost all of it is focused on fixing programming languages. But given how coding has changed in the last decade or so, I’m wondering if focusing some of their efforts on libraries and/or frameworks would make more sense (WARNING: Very Geeky).

Take data science. Many say that the Python programming language has become increasingly important in data science, but that’s not really accurate. When people say “Python,” what they really mean are several libraries written in Python: tensorflow or PyTorch for machine learning/AI, pandas for slicing and dicing data, etc. And if you’re going to visualize your data in pandas, you need to master one of several Python data visualization libraries. Or you might use d3, a data visualization library written in JavaScript that’s responsible for a lot of the gorgeous visualizations you see on major news sites.

Even more importantly, having taken an intro class in Python doesn’t really prepare you to work in these libraries. Both pandas and d3 use an approach that’s called “functional programming,” and as I can attest from teaching them, they both look and feel wildly different than working in vanilla Python/JavaScript.

Similarly, the modern world of JavaScript web and application development — e.g., Node — has morphed into something that’s unrecognizable to someone who’d learned JavaScript many years ago. Even worse, there’s a remarkable amount of churn. If you’d taken the time to learn the Angular application framework a few years ago, by the time you’d really mastered it you’d be under a lot of pressure to switch to newer frameworks such as React or Vue.

And that’s a real problem for beginners: they can’t do a heck of a lot of useful work unless you’ve also mastered one or more libraries/frameworks. Let’s take one more example. Say you are a beginner who’s gotten excited about programming in AR/VR. To do it, you’d have to both learn a language and a framework — either C# and Unity or JavaScript and A-Frame (my favorite). Based on my limited experience learning/teaching them, wrapping your head around the frameworks, not the underlying language, is the biggest challenge for beginners. And as I’ve discussed previously, the stumbling blocks a beginner faces working in a framework may be very different than those for a language.

Finally, libraries/frameworks also offer one big advantage over languages for researchers who are trying to have an impact on the real world: as far as I can tell, most programmers switch libraries/frameworks more frequently than they do languages. Getting an ecosystem of Python coders to switch to a new language designed by researchers to be easier to learn? That’s a pretty tall order. Convincing them to switch from one Python data visualization library to another is a much easier sell. And in the JavaScript world? Folks have gotten so used to a ridiculous rate of framework/library churn that if a researcher created a new framework that was substantially easier to use, odds are good that a lot of coders would jump ship.

Obviously there’s still plenty of room to do interesting research on making languages easier to learn and use. But if researchers want to have a bigger impact, libraries/frameworks might be a better bet.

Are Orgs Like Uber Creating a New Workplace or Reviving an Old One?

I’ve been reading about the political economy of late 19th century US, and a quote from a recent article by historian Charles Postel caught my eye.

The advocates of the new work regime claim that it is paving the way to the “flexible” economy of the future. In the process, it is unleashing the inner entrepreneur of the Walmart “associate” and the Uber “peer.” But for millions of workers the new “flexibility” means little more than overwork and insecurity, and the advertised work regime of the future represents a throwback to the exploitative tyranny of the first machine age.

In the late 19th century, the telegraph, steam engines, and electric power changed everything. But they also changed nothing. Because, as before, most work involved long hours at low pay in domestic service, farm labor, construction, mining, and other strenuous jobs. Hiring was often day-by-day, and many workers operated as semi-independent contractors. The infamous “sweating system” meant families set their own hours, their own pace, in their own living spaces (tenements). Coal miners, too, often worked as their own bosses, getting paid by the ton. This “flexible” work regime translated into hazardous work, child labor, and physical and mental torment. Workers stood one accident or bout of unemployment or sickness from catastrophe.

Uber may be using an app to manage the people who do its work but the people infrastructure “stack” the app sits on is right out of the 1880s.

The tech world is skilled at making us believe that everything it’s selling is new. And because change is occurring at a blistering pace, it’s easy to assume that when we compare the present to the past it is “disrupting,” the only history we need to consider is the history of Right Before Now. It’s not.

The work arrangements of Right Before Now, where many people had one employer who provided steady employment and benefits, wasn’t natural or inevitable. People fought and bled to create it, to “disrupt” the previous people infrastructure stack so children could have a childhood and an unprecedented number of working families could benefit from the prosperity they helped create.

In short, let’s not get suckered by companies like Uber. Let’s stop talking about “disrupting” and start talking about what values we want the new world of work to uphold.

By 2030, the Global 1% Will Own Two Thirds of All Wealth

According to the Guardian newspaper, we’re likely headed towards an unparalleled concentration of global wealth.

An alarming projection produced by the [UK] House of Commons library suggests that if trends seen since the 2008 financial crash were to continue, then the top 1% will hold 64% of the world’s wealth by 2030

The reason for this disturbing prediction? Wealth was already far too concentrated before the financial crashes of 2008. And since then,

the wealth of the richest 1% has been growing at an average of 6% a year – much faster than the 3% growth in wealth of the remaining 99% of the world’s population

And as far as I can tell, this prediction doesn’t take into account the potential impact of robots/AI, which sure looks like it’s going to create massive increases in inequality if we don’t change the rules of the road.

If we want our democracy to survive the age of robots/AI, figuring out what to do about jobs isn’t enough; we also have to confront head-on the concentration of wealth.

Writing Our Stories, Writing Code: A Community-Centered Approach to Teaching Coding

While working on the Mixed Reality for All project, I started thinking about how you might teach coding to adults in low income communitities where instead of treating them as isolated individuals, you’d take advantage of the power of community. At one point I wrote a quick-and-dirty sketch of what such a class might look like. It’s pretty rough — for example, I had some ideas about how you could take advantage of existing instituions, such as churches or unions, but didnt have the time to think them through. But there are enough ideas here that I think it’s worth sharing.

Google has a VR development kit called Google Expeditions that lets teachers teach through stories by creating “immersive, virtual journeys” for their students. It’s pretty impressive. But when using Google Expeditions, teachers are limited to just the tools that Google decided to put in this toolbox. If teachers need more tools to tell their story, they’re out of luck.

What if people in low income communities could create their own stories in VR/AR, and instead of being limited to the tools in the toolbox a big corporation gave them, they could make new tools? Could they unleash their inner Grand Master Flash, who helped create Hip-Hop by morphing a tool for playing music — a turntable — into a tool for making music?

Or to put it another way: what if we can interweave the art of creating stories in AR/VR with learning  the craft of how to code?

It might look something like this:

In the first workshop, a group of adults meet for a Friday night and half of a Saturday. On Friday night, first they participate in a story circle. Then they learn how to use one simple coding technique to begin the journey of expressing their story in VR. For example, each participant comes up with 3 words that sums up their story. Using a code template that displays one word in VR, they create their first VR page that displays their 3 words (see example at the end). They finish by showing off their VR page and do a brief check-in about how they’re feeling about what they’ve learned so far.

On Saturday, they start adding a few tools to their storytelling toolbox — e.g., tools for adding a picture, a paragraph of text, some simple interactions and animation — so they can create the first version of their story. The workshop alternates between a little instruction, a lot of playing and experimenting with code — possibly working in pairs ala pair programming — and thinking together about storytelling & reflecting on their experience so far. Through play, they practice the same coding techniques over & over so coding starts to feel less scary and more like a means of expressing themselves.

The group meets again for a few shorter Saturday sessions that take place every other week. Each time they learn one or two more coding techniques, a little more about VR/AR design & how to tell a story, plus sharing their stories about the experience. In doing so, they also build the trust & community they need to help them get over any fears (which is at least 50% of the battle).  In between these sessions, they work on their own or with coding buddies on their coding skill and their story — and each week the main group doesn’t meet, there is an optional Saturday drop in session for anyone who needs a little help.

Then they start the 2nd part of the course: learning how to make tools to add to their toolkit. They begin with another Friday night – Saturday half day workshop, then meet every other week for shorter Saturday sessions. Although this part will be harder than Part 1, they will already have a solid grounding in some of the basic skills they need to do this, and they will have enough experience of thinking of coding not as this foreign thing that only super geeks can do but a way of expressing themselves. As a result, rather than each new technique being a chance to feel stupid, it’s a chance to expand how they can express themselves. And they will have a community of trust — a “Band of Brothers and Sisters” — to help them get through any parts that feel intimidating/scary.

In these sessions, they also begin to discuss what it might mean to make an economy where more and more people could make part of their living by making and sharing tools and other ways of creating value, wealth, ownership, and community in AR/VR. For example, what would it mean to say “we own what we make” in this new economy?

By the end of this part of their journey, those who want to continue should be able to start meeting on their own, getting help through the network they have already learned how to plug into of other folks around the country who’ve gone through a similar experience and who — with occasional help from world-class expert techies from around the globe —  have been helping to shape the path people take to keep improving their skills. And perhaps a few of them will learn how to help teach the next set of workshops.

Here’s what the augmented reality example above might look like:

Although you don’t have a virtual reality headset, you can still view and interact with these pages using a desktop web browser– either Google Chrome or Firefox. To move around, use the arrow keys to move forward and back and your mouse to rotate what you’re seeing. You can also view them on a smart phone, although on my iPhone I need to tilt it up before I can see the words.

Many years ago I taught adult education workshops on beginning HTML, and I’m pretty confident that anyone with a high school level literacy could be taught to be comfortable changing the one word template into their own 3 words and to understand what they’re doing. The key is a) starting with a group exercise that lets people understand what it means to code (my favorite: “the boss’s idiot nephew”)  and b) lots & lots of practice with supportive people.

Every Move You Make, Every Breath You Take, Big Tech Will Be Watching You

I think we can now confidently say we’re now at the point where Big Tech needs to be seriously regulated. Facebook is in the hot seat right today, but as New York Times research into Amazon and Google patents show, there’s plenty more to be nervous about.

A scene from a future Amazon hopes to bring about, as described in a November patent:

You’re taking to a friend on the phone or “within a detectable distance of [a] device” like Alexa. You say

The vacation was wonderful. I really enjoyed Orange County and the beaches. And the kids loved the San Diego zoo.

The device is running a “sniffer algorithm” that’s looking for “trigger words,” “often a verb indicating some level of desire or interest” such as like, love, enjoy, downloaded, hate, returned.

For each identified potential trigger word, the device can capture adjacent audio that can be analyzed, on the device or remotely, to attempt to determine one or more keywords associated with that trigger word.

So when you said “loves,” it triggers the device to analyze your conversation and extract the words “San Diego zoo.” It’d does the same thing for your friend, who replied,

When we went to Southern California, I fell in love with Santa Barbara. There were so many great wineries to visit.

So at the end of the call, your device could send you a discount for a San Diego season zoo pass and her a wine of the month discount. And all of the info about these keywords, including the context of the call, could be stored as part of a profile of you and your friend.

Google’s even more ambitious. They don’t just want to extract info from every conversation you have, they want every drop of info they can get. In their September 2016 patent about a home monitoring system, they envision the following, either in a homeowner’s— or renter’s— home could be closely monitored. According to the patent,

an audio signature matching a dining chair movement across a floor may suggest that an occupant is sitting in the chair (e.g., because the occupant may have presumably moved the chair to sit in it). Indeed, video inputs may confirm and/or identify that occupants are sitting in the chair and/or at the table. Additionally, smart device inputs may be used to obtain a number of contextual clues, such as utensil movement, conversation content, vapor detection, etc. For example, in one embodiment, the vapor sensors may detect the presence of food within the dining room zone, which may indicate that a meal is being consumed in the dining room….. an audio signature of keyboard clicking, a desk chair moving, and/or papers shuffling etc. may indicate that someone is working…. an audio signature and/or video signature may be associated with the sounds and/or images of teeth brushing in the zone 618. Next, additional characteristics may be determined (e.g., the sink being left on, a duration of teeth brushing, a speed of teeth brushing, etc.). These findings may me reported and/or recorded within the system (e.g., for subsequent control and/or reporting by the household policy manager [program ]) .

And all of this info could be sliced and diced by demographic info.

Demographic information may include, for example: occupant information such as: number of occupants, gender of occupants, age of occupants, ethnicity of occupants.

And the techniques for gathering this data could be pretty sophisticated.

By way of example, a video monitoring camera placed in the kitchen of the home can perform image processing on several days or weeks worth of captured data to determine how many different individuals it sees on a regular basis, to establish how many occupants live in the house.

What could be done with all this data? Well, the patent suggests, you could use it to identify children’s behavior.

For example, characteristics of audio signatures, such as speech patterns, pitch, etc. may be used to discern child occupancy. Next, the occupants may be monitored, specifically listening for low-level audio signatures (e.g., whispering or silence), while the occupants are active (e.g., moving or performing other actions). Based upon the detection of these low-level audio signatures combined with active monitored occupants, the system may infer that mischief (e.g., activities that should not be occurring) is occurring….. For example, it may be expected that certain activities be performed in quiet, thus indicating that the quiet activity is unlikely to be mischief. For example, reading a book, mediating, etc. are oftentimes performed in quiet…..

For example, audio monitoring, optical monitoring, infrared monitoring, etc. may be used to discern occupancy and undesirable activities of the occupants. In one embodiment, the contextual data may include audio signatures indicating “bully” keywords such as derogatory name-calling, elevated voices, etc. Accordingly, the system may monitor for and detect the use of such “bully” keywords. Additionally, in some embodiments, the contextual data may include audio signatures indicating the use of foul language.

And not just kids.

returning to the “I’ll be home by 5:00” assertion, the system may determine a current location of the household member and how long it would take to get from the current location to the occupant’s house (block 394 of FIG. 46). If the occupant’s house can be reached by 5:00, a determination is made as to whether the assertion/promise has been met (decision block 356 of FIG. 45 and decision block 400 of FIG. 46)…. If the assertion has been met, routine monitoring proceeds (block 358 of FIG. 45 and block 402 of FIG. 46)…. If the assertion/promise cannot be met (e.g., the occupant’s house cannot be reached by 5:00), a finding that the assertion cannot be met may be reported and/or recorded (block 360 of FIG. 45 and block 398 of FIG. 46).

What an abusive spouse — or an authoritarian regime — could do with this kind of automated micro surveillance is not a topic covered by the patent.

The system could also be used to monitor group behavior:

For example, if the household indicated a goal to spend more time doing activities together and the system monitoring indicates that the household is spending less time together or marginally more time together, the system may provide a reminder of the household goal to one or more members of the household (e.g., via an audible and/or visual alert in the household, via a text message provided to the user’s smartphone, etc.)…. If sufficient progress toward the goal is attained, a progress reward may be provided to one or more members of the household. For example, in the togetherness goal mentioned above, if the family spends 20 additional minutes together in a week, when the goal is to spend an additional hour together a week, a progress reward may be provided to the family.

And you could use it to see if you’re keeping up with the Joneses.

In one embodiment, household bragging rights may be a reward, by providing a neighborhood message to other participating households, stating that the household is progressing towards and/or has attained the particular goal.

These visions of a world in which Big Tech has the ability record any scrap of info about us aren’t from some random email an idiot marketing manager sent to their boss or videotaped off the cuff remarks at some conference. At least a few people at Amazon and Google thought they were nifty enough to apply for patents. That’s not remotely the same as official business plans. But the fact that there’s a corporate culture at places like Amazon and Google where these public legal documents would be considered OK should give us pause.

There are a lot of great, caring people who work at Big Tech companies who’d be appalled at the iPanOpticon sentiments behind these patents. But given the resources, power, and reach that Big Tech now has, we need to make damn sure that while they “move fast and break things” they don’t end destroying our liberty and our society’s future.