Look Out (AR) World, Here Comes Google

Google steps up its AR game with ARCore:

A fabulous, whimsical ARCore experiment by Jane Feidhoff (more info here).

Tilt Brush was built for VR, but apparently it will work with Google’s new ARCore universe.

To help refine Tilt Brush, Google set up an Artist in Residence program that eventually involved a total of 60 artists from a wide variety of styles:

At some point I’ll do an analysis of Google’s new venture into AR. For now I just wanted to post these videos, because they’re so freakin cool.

On AI and Not Sitting At the Back of the Bus

On “Deep Learning” AI and Not Sitting At the Back of the Bus

In my post on the emerging tech of augmented reality (AR), I argued that the AR offers community groups — especially those in communities that our society has written off — a unique opportunity to change the course of emerging tech so that more the benefits flow to their communities. I think the same is true for AI.

As I mentioned in an earlier post, robots aren’t ready for ordinary folks to do real work: the tech is still too primitive. But a lot of robotics will be driven by technology where the baby version of it is available now. Most notably, although AI is ridiculously underpowered compared to where it’ll be in 20+ years, you can actually do something useful with it today — and the field is really taking off. Forrester predicted that investment in AI would grow by 30% this year. Gartner predicts that “by 2020, AI technologies will be virtually pervasive in almost every new software product and service,” and IDC estimates that by 2020 revenue from AI-based systems will hit $47 billion. There is also an almost insatiable demand for people who are fluent with current AI tech.

As a result, Google, Microsoft, and others are in a race to see who can make their AI programming libraries and tools more accessible. Today, a typical programmer could learn how to write a 15-line program that will, say, recognize pictures of a cat or answer a moderately impressive range of questions that only advanced practitioners could accomplish a decade ago (see more at the end of this post). Becoming really skilled at this tech still takes a lot of work, but the barriers to entry are far lower than they used to be.

There’s also been a flurry of work online to make it easier to learn these AI tools. For example, Siraj Raval has created a charming, funny YouTube series aimed at hackers who want to do cool stuff with AI; Josh Gordon at Google and Brandon Rohrer at Microsoft (Microsoft: whole course) have also produced really helpful YouTube tutorials. And there are first-rate free courses at Fast AI that make it much easier for someone with some programming background to do some really impressive work.

Almost all the major players in this space they want to “democratize” this technology. Recently, for example, Google launched the People and AI Research Initiative PAIR), whose goal is

“to focus on the “human side” of AI: the relationship between users and technology, the new applications it enables, and how to make it broadly inclusive.”

Based on what they’ve done so far, I think the major tech players are very sincere in wanting to make AI more accessible.

But when these players use the word “democratize,” what their work shows they really mean is making it fully accessible to people like me, who are programmers. There are more and more types of analysis that you can do using AI by pushing a few buttons. But once you go beyond basic work, there’s a pretty big gap between what most tech players they want to do and what they are doing.

That’s why I think a network of community groups could have a real impact on AI. And unlike just a few years ago, there are a lot more opportunities for some interesting partnerships.
For example, fast.ai’s terrific courses say they are aimed at “anyone with at least one year’s coding experience.” If any network of community groups wanted to work with fast.ai to create courses and a community-oriented ecosystem for learning that would make this knowledge accessible to even more people, I’m sure the people at fast.ai would be excited by the idea.

Playing with AI today isn’t as fun as playing with augmented reality. But there are some incredible opportunities here to help shape the direction of the tech industry and who will benefit from this emerging tech.


NOTE: If you are one of the readers of this blog who likes geeking out with tech and want to get your feet wet with AI, I’d recommend playing with Keras. Keras is an open source library developed by a someone at Google which makes coding “deep learning” AI much easier. Through Keras, you can work with a variety of AI libraries, including Google’s Tensorflow and Microsoft’s CNTK, using the same code. The other fabulous thing about Keras is that it bakes in a lot of best practices. When I first tried to start learning machine learning and AI a few years ago, figuring out how to write the code wasn’t that hard. But to properly use the code, you needed to make decisions about a bewildering number of options, and trying to figure out what those options meant was very painful. Keras sets a lot of those options for you, see you don’t have to worry about tweaking them until you really know what you’re doing.

If you do decide you want to play, I would strongly recommend starting with Fast AI‘s Practical Deep Learning for Coders. It’s a very impressive piece of work. And in the first hands-on lesson, you learn how to do some really cool stuff in just a handful of lines of code.

App Stores, Streaming Music, and Community Voice: A Thought Experiment

My post on community groups and augmented reality (AR) covered a lot of ground. Today I’m going spend a little more time on one point: what could this sort of organizing accomplish?

So, a thought experiment:

Imagine that if before Apple created its iPhone apps store, instead of making decisions unilaterally, it was in dialogue with communities from Compton to Letcher County, Kentucky to the suburban Rust Belt of Ohio — people who by the Internet Age our society had effectively written off. And imagine that people from these communities had a voice not only because they could vote or organize economic pressure but because in each of these communities many people were fluent with the tech behind iPhones.

Now imagine that if before Spotify, YouTube, and other players “disrupted” the music industry, transforming an already unjust, racist system so that now most musicians can’t earn a living by selling/streaming their records, these same communities had a real say in shaping the new rules of the road and who benefited from them.

Now take that impact and multiply it by 100. Sometime in the next 10 to 20 years we are going to reach a point where similar decisions are being made about how robots/AI will dramatically reshape our economy. Only this time the stakes will be much higher: a staggering amount of wealth will be created, and it’s possible that an unimaginable number of jobs may be destroyed.

That’s why it’s worth figuring out how all communities can have a seat at the table where the rules of this new road are made.

In turn, that’s why it’s critical that in each of these communities there are enough people who are working with this new tech and can explain them to the rest of their community. A seat at the table is meaningless if you don’t understand the nuances of this new tech well enough to understand a rule’s implications. And often the table where community groups need a seat won’t be in Congress, it’ll be at meeting of coders.


NOTE: Just in case it wasn’t clear, the Apple and Spotify/YouTube examples really are just thought experiments. For example, in the real world it wouldn’t be fair to expect that Apple would’ve shared enough of their iPhone plans with the public in advance so that communities could have had a real say. What I’ve learned from talking with people about Makers All is that it’s hard for most folks to imagine what an economy might look like when robots/AI are ubiquitous; these thought experiments are a small step in helping folks wrap their head around it.

Why Augmented Reality Instead of Robots?

One point missing from last week’s post on how a network of community groups could influence the direction of Augmented Reality (AR): why start with AR? Why not start with robots?

The answer is simple: robots aren’t ready for ordinary folks to do real work. The tech is still too primitive; it’s too difficult to do anything useful with robots unless you’re a pro. Here’s Roomba inventor Joe Jones on how far robots have to go:

Robots are deceptively hard. My co-founder at Harvest, Clara Vu, likes to say, “A robot application that a technology grad student thinks they can easily do in a couple of weeks has an outside chance of actually being practical. Anything harder is impossible.”

In contrast, AR is good to go. In the next few years it’ll get easier to do development in AR, but even today there’s plenty of room for community based efforts to do interesting work in AR.

In doing so, this network of community groups would lay the technical groundwork needed for the coming robot revolution. Augmented reality and robots don’t use exactly the same tech. But if you know how to do advanced AR coding, picking up how to program robots shouldn’t be too big of a stretch — especially since a lot of the programming languages for machine learning/AI that will eventually play a critical role with robots can be picked up today.

Similarly, once this network’s trainers have figured out the techniques, processes, metaphors, etc. to make the coding concepts behind AR much more accessible, they’ve got a blueprint for doing so with robots much more quickly. Having already proven out these techniques with AR, it’ll be much easier to convince the designers of robot software to incorporate some of the techniques from the get-go.

More importantly, the work around AR will help build the networks of relationships and trust within and across communities. As anyone who’s been involved in multi-community organizing can tell you, building this kind of social infrastructure takes a lot of blood, sweat, tears, and time (not to mention repeatedly banging your head against the wall). But once those relationships and trust have been built, they can easily be reused when the robot tech is ready.

As a result, by the time robots had developed to the point where, for example, it was clear that in a year or two we would start to see robot app stores, there should be more than enough of the community infrastructure these groups would need to successfully win a seat at the table.

On Augmented Reality and Not Sitting at the Back of the Bus

Cutting edge tech usually takes a while before it reaches low-income or marginalized communities. Hard-working community groups often struggle to train a handful of people to use this new tech so that a trickle of the billions of dollars generated by this tech reaches their community.

But what if instead of being relegated to the back of the bus, these communities grabbed a seat at the front?

To understand what that might look like, here’s a thought experiment using the cutting-edge field of augmented reality (AR).

If virtual reality is about creating objects in an imaginary world, augmented reality is about placing imaginary objects in the real world. Last year’s Pokémon Go craze was an amazing albeit goofy example of one way augmented reality can work: looking at the world through your smart phone, you can play Pokémon in the real life. In the next few years, augmented reality is going to take off like a rocket, and there are billions of dollars of profit at stake.

Next month when iOS 11 is rolled out, ARKit, Apple’s opening salvo in the battle over these profits, will turn newer iPhones into a tool for general purpose augmented reality:

ARKit lets developers build AR apps, which integrate digital experiences into the physical world via iPhone or iPad, a la Pokémon Go. Those apps will be available to consumers when iOS 11 arrives in September. But developers have started tinkering—creating tools that let you see how furniture fits in a room or quickly calculate the area of your kitchen…. Matthew Miesnieks, a VC who led a team researching AR within Samsung, calls ARKit “the biggest thing that’s happened to the AR industry since it began,” and he’s not alone in his enthusiasm. By getting AR in the hands of millions of iPhone users, Apple is poised to become the world’s most powerful and popular purveyor of augmented-reality apps. And by opening up its developers’ kit, it’s powering hundreds of experiments into what, precisely, this medium is good for.

Already we’ve seen examples of using ARKit as everything from using it as a measuring tape to populating your room with dogs and space cats to a really cool interactive music video. Industry analysts also speculate that iPhone-based AR is just Apple’s first step and that they are developing headsets which will provide a more immersive experience.

Meanwhile, Microsoft is so confident about their Hololens, the augmented and virtual reality glasses they started rolling out to select partners last year, that it’s going to skip releasing version 2.0 this year and instead is going release a souped-up, AI-infused v3.0 to the public in 2019. Hololens is already being used in the real world. Thyssenkrupp already has 100 elevator and stair lift technicians testing it out in the field as an AR-driven tech-manual-plus-Skype tool for elevator repair. Case Western is experimenting with using Hololens for creating museum walk-throughs. And when Case Western and the Cleveland Clinic open a new health education campus in 2019, students will learn anatomy using a Hololens AR cadaver.

Because of what’s known as the network effect, battles over emerging tech are almost always winners-take-all. So the titans of the tech world are now in a fierce battle to see which 2 or 3 players will dominate this multi-billion-dollar industry for years to come. As a result, there is a unique opportunity for community groups to influence the outcome — if they work together.

What if groups in communities from Compton to Letcher County, Kentucky to the suburban Rust Belt of Ohio created a network to ensure that as this new tech started taking off, their voices were heard and their communities were involved in the work shaping the course of this new tech and its economic impact?

To make that happen, they could leverage a problem most tech players have. One of the main lessons of my decades of experience in both the corporate and community world is that access isn’t just an issue for marginalized communities; corporations and large nonprofits also struggle with it. Corporations can often paper over the problem by throwing money at it, spending scary amounts of money on consultants and/or hiring just enough techies to do work that regular staff ought to be able to handle if the tech wasn’t so hard to use. But if this network of communities can experiment with the new tech, they should be able to show the major players how to change the tech so it’s also far more accessible in the corporate world. If it’s much easier for people in South Central or Harlan County to use new tech without being uber-geeks, it’ll certainly be a lot more accessible to corporate “power users”– people who aren’t trained programmers but who can, say, make Excel sit up & dance and who often play a critical role in the diffusion of new tech through a large organization. And that’s a huge competitive advantage for any tech players striving to dominate this new tech.

To make this emerging tech more accessible, the community network would need to focus on two tasks:

  • Making the existing tools for this new tech more accessible by creating better trainings and, even more importantly, creating an ecosystem around these tools that provides longer term support. A one-time training is a good place to start, but to gain fluency people need the ability to easily get help as they continue to use the tech.
  • Doing simple experiments to figure out how the tools could be remade to smooth the learning curve. With most new tech, it’s really easy to do a handful of tasks, but as soon you want to go beyond the basics you fall off a cliff. Even corporate power users will often end up wasting an absurd number of hours banging their head against the wall to get this tech to work. Depending on how this new tech is designed — e.g., if it uses an “open source” approach where people can modify the code themselves — it may even be possible to use these experiments to either make the tools easier to use or to build a layer on top that hides some of the more confusing aspects.

But how could these small community groups, who are already severely strapped for resources, have the time and energy to make this happen without being diverted from the important work they’re already doing? By a second act of leveraging: collaborating with techies outside of their groups.

Odds are, for example, there are students in colleges near some of these community groups who would love to help but don’t know how. Similarly, there web designers who would love to learn how to work with augmented reality, which is going to dramatically change their world, and are also passionate about helping the community. There are programmers who’d love to play around with coding augmented reality, both for professional development but also because it’s their idea of fun. The network of communities would give these techies an easy way to have an impact on the world that would be deeply satisfying.

The same thing is also true for experts in the field. One of the cool things about the tech world is that many of the most skilled people in it are remarkably generous with their time. They love the tech and want everyone to be able to enjoy it. Again, the hardest problem many of them have is that they simply don’t know where to begin. They strongly believe in “democratizing” the tech but lack the skills to take this complex knowledge and break it down for everyday folks. But in most communities that would be part of the network, there are people who may not be techies but who are experts in breaking down complex concepts so they make sense to folks in their community.

Best of all, many of the techies who would love to help are themselves part of existing local & national networks that the network of marginalized communities could tap into.

In short, this network of communities could use a “stone soup”-style organizing approach, helping bring together far more resources than they normally could. And that in turn should make it pretty straightforward to get tech players to want to work with them.

But how could groups who are part of this network afford to get their hands on this new tech? If communities are working together, I think that’s a problem that’s pretty straightforward to solve. When the tech is very early, the tiny startups who are driving it forward don’t have the money to donate equipment or sell it dirt cheap. But when the tech hits the stage where tech titans are battling over who will dominate it, these players certainly do. In fact, if they think of it as early market penetration plus great PR, seeding this equipment in these marginalized communities is really just a form of cheap advertising — and these are folks that are going to end up spending an awful lot of money on advertising given the profits that are ultimately at stake.

To people who haven’t had the experience of being involved in emerging tech, this may seem crazily ambitious. But one of the most important lessons of the last decade with new tech is that small groups can have a remarkable impact if instead of trying to build perfect palaces, they iterate using lots of small experiments. The key is to take a low-risk approach: start with a handful of people trying out the new tech, explore & map out opportunities, and then take baby steps. The hard part here isn’t the tech, it’s the people: getting a network of community groups to strategically act together in a new domain is not a trivial task.

In almost every community that our society has abandoned, you’ll find amazing people doing what they can to help their communities get back on their feet. Up till now, it’s been a battle fought on terrain where it’s incredibly hard to win. Maybe it’s time to try some experiments fighting on the terrain of emerging tech.

Up Next: a similar strategy for artificial intelligence, and why focus today on augmented reality rather than robots


UPDATE: two weeks after I wrote this post, Google announced their next big step in AR. Their earlier work was very interesting but aimed at a relatively small set of devices; their newest entry is going for a wider audience, and it’s very impressive.

About Those 7-11 Jobs…

This summer, unmanned convenience stores are taking off in Beijing and Shanghai. BingoBox, Eat Box, and others are launching stores where you grab, pay, and go without ever talking to a clerk. 

 To keep you from just grabbing and going, these stores have a double door-based security system where you to use either a QR code or facial recognition tied to an account to get into the store.  When you try to leave, you can’t get through the second door if the RFID tag on your soda doesn’t tell the security system that you’ve paid.

Nor surpassingly, the launches haven’t gone off without a hitch. BingoBox stores discovered the hard way that if you don’t have good air conditioning, the summer turns their store into a HeatBox, melting chocolates and little cakes. But overall, their prospects look very promising.

I couldn’t find estimates on how much these unmanned stores will reduce labor costs; even if there’s no clerk, someone still needs to restock, keep the place clean, etc, and from what I could read it’s not clear how the stores are getting that work done. And of all the jobs that may get eliminated, I don’t think many people are going to shed too many tears over the loss of these jobs given how poorly they pay and how crappy these jobs are. But as one more sign of where we might be headed — especially given that the tech involved in eliminating these jobs is pretty primitive — it doesn’t  paint an optimistic picture.

For more details, check out this walk though of an Eat Box: