Disney’s Unique Take on Diversifying Its Coders

What you do have your big corporation and you want to create more diversity in your IT team? According to Fast Company, Disney has a new twist on an old strategy: offer tech retraining and apprenticeships to female “employees already well into their careers.” The program, which was created by Disney VP of Technology Nikki Katz, is called CODE: Rosie.

The CODE in CODE: Rosie stands for Creating Opportunities for Diverse Engineers. The “Rosie” part references Rosie the Riveter, the symbol of World War II’s working women; an internal CODE Rosie logo even depicts Minnie Mouse in Rosie’s iconic rolled-up-sleeve pose. In particular, the program pays tribute to the “Rosies” who programmed the U.S. Army’s pioneering ENIAC computer back in the 1940s.

For decades after those ENIAC coders helped make history, women played prominent roles in software engineering. But the percentage of U.S. computer-science majors who are female peaked more than 30 years ago and then spiraled downward for years. In recent years, groups such as Girls Who Code, Black Girls Code, and BRAID have worked to get young women interested in programming early, in hopes of putting them on a track to pursue an education—and then a career—in computer science. These efforts have helped, but the field’s gender imbalance remains severe.

CODE: Rosie has a couple of features that make it unique. First, it’s got a strong emphasis on apprenticeships inside the corporation.

After three months of training—in everything from basic computer-science concepts to programming languages such as Python—they’ll segue into a yearlong apprenticeship consisting of two six-month chunks in different teams within the company. Then they’ll have the opportunity to take a job within one of Disney’s technical groups.

Second, to ensure that people who join the program understand what they’re getting into, are committed enough to it that investing in them will pay off, and to give them a leg up, CODE: Rosie requires that participants do a bunch of work before they are accepted into the program.

for the first CODE: Rosie program, which kicked off in April 2016 and had a dozen available slots, applicants had to submit essays and tackle a simple coding project…. When Katz and her colleagues were formulating this year’s “CODE: Rosie 2.0,” however, they decided to front-load 40 to 60 hours of online instruction into the application process. That makes it much more of a gauntlet, especially given that a prospective Rosie must go through this material on her own time…. Along with allowing employees to show they’re serious about the CODE: Rosie program, this pre-training helps them move ahead more quickly once they’re in.

Third, it provides a strong safety net that reduces the risk to participants if the program doesn’t work out for them.

Disney holds particpants’ previous non-technical jobs open and gives each “Rosie” the option to return to her old role rather than continuing on her new career path. Of the 12 Rosies in the inaugural program, only one took the company up on this offer. (Another ended up leaving the company for a technical job elsewhere.)

Fourth, Katz invested in building strong support for the program from key techies.

To ensure that Rosies would be welcomed, Katz worked to get advance buy-in from Disney’s tech pros. “I got this lovely email from Nikki, only a few months after hiring into Disney,” remembers lead software engineer Calvin Wong. “It was basically describing the CODE: Rosie program: ‘We’re going to take a lot of candidates, teach them a lot of software engineering skill sets, and then hopefully let these candidates explore a new career path.’” Along with other employees, Wong became a CODE: Rosie buddy, responsible for both answering day-to-day questions and providing ongoing mentorship.

Wong has found that role to be rewarding, in part because the Rosies aren’t seasoned engineers. “From high school all the way until now, I’ve only being doing tech,” he says. “Having them come into the program and talk about their past experiences in consumer products and game development and all these other fields was just really eye-opening. Because I only saw the Walt Disney Company in a narrow spectrum.”

It’s not clear how big an impact CODE: Rosie will have. It’s only in its second year and it’s still very small — and it’s still too early to tell where it will go.

In a company with almost 200,000 employees around the world, a 20-person training effort like CODE: Rosie 2.0–which is open only to Los Angeles-area staffers–can’t accommodate everyone who might benefit from it. Katz emphasizes that the program is a “white glove, boutique” undertaking and there’s a limit to how far future versions could scale up. But she quickly adds that the undertaking hasn’t just changed the lives of the Rosies who get in—it’s also changed Disney.

“When you do something authentically, for the right reasons, that is maybe a little different from the way we’ve tried things before, it tends to have these ripple effects in the organization.” she says.

Although it’s still at the very early stages, I think there may be some valuable lessons here not just for large corporations but for how we think about retraining whole communities. More on that in a future post.

Can Babershops Help Save Us From Job-Killing Robots?

Interesting news from a new study on reducing blood pressure. A few years ago, researchers had trained barbers “to check blood pressure and refer people with high levels to physicians.” It only had a small effect, they thought, because too often the physicians didn’t prescribe blood pressure medication. So this time, researchers tried a different approach:

The control group consisted of barbers who encouraged lifestyle modification or referred customers with high blood pressure to physicians. In the intervention group, barbers screened patients, then handed them off to pharmacists who met with customers in the barbershops. They treated patients with medications and lifestyle changes according to set protocols, then updated physicians on what they had done.

They also tweaked the process to handle the inevitable bumps in the road.

When barbers weren’t consistently screening customers by measuring their blood pressure, pharmacists stepped in to do that. When labs slowed things down, pharmacists brought measuring tests to the barbershops.

The end result:

Six months into the trial, systolic blood pressure (the higher of the two blood pressure measures) in the control group had dropped about 9 mm Hg (millimeters of mercury) to 145.4, which is still high. In the intervention group, though, blood pressure had dropped 27 mm Hg to 125.8, which is close to “normal.” If we define the goal of blood pressure management to be less than 130/80, more than 63 percent of the intervention group achieved it, compared with less than 12 percent of the control group.

It gets better. The rate of cohort retention — measuring how many of the patients remained plugged into the study and care throughout the entire process — was 95 percent.

According to the New York Times, the researchers believed there were several important lessons that could be learned from the results of this series of studies.

Getting barbers involved meant health messages came from trusted members of the community. Locating the intervention in barbershops meant patients could receive care without inconvenience, with peer support. Using pharmacists meant that care could be delivered more efficiently.

I think there’s also a valuable lesson here for Makers All. If emerging tech is going to provide real opportunities to communities that our society has basically written off, we need to plug into the parts of that community that people already trust. Learning complex new tech can be really intimidating, especially because you’re probably going to spend a lot of time falling on your face when you first get started. Feeling safe and knowing there are people who’ve got your back is half the battle. Barbershops and beauty salons, churches — all of these institutions could play an important role.

Community institutions aren’t magic. As the study also showed, it will take the right mix of institutions as well as a feedback loop to tweak the process as we learn what’s working and what’s not. But if we’re smart about how we do it, a community-oriented approach has tremendous potential.

Model the Blue-Collar Coding Job Ladder After Restaurants, Not Construction Work

A few years ago, Anil Dash wrote an influential piece advocating for what he called “blue-collar coding”:

High schools have long offered vocational education, preparing graduates for practical careers by making them proficient in valuable technical skill sets which they can put to use directly in the job market right after graduation. Vocational-technical schools (vo-tech) provide trained workers in important fields such as healthcare, construction trades, and core business functions like accounting. For a significant number of my high school peers, vo-tech was the best path to a professional job that would pay well over the duration of an entire career. Now it’s time that vo-tech programs broadly add internet and web technologies to the mix. We need web dev vo-tech…

Put another way, our industry can grow in a very meaningful way by giving lots of young people at a high school level the knowledge they need to learn jQuery straight out of high school, or teaching maintenance on a MySQL database at a trade school without having to get a graduate degree in computer science. That’s not to say that CS students aren’t also important — we’ll need the breakthroughs and innovations they discover. But someone has to run that intranet app at an insurance company, and somebody has to maintain the internal iOS app at a law firm, and those are solid, respectable jobs that are as key to our economy as a 22-year-old trying to pivot and iterate their way into an acqu-hire.

I think this is really smart. The one qualm I’ve had about this idea is that when people use the phrase “blue-collar coding,” there’s often an implicit assumption that blue-collar coding jobs would be silo’d off from the highly skilled developers who create the software, frameworks, etc. that everyone else uses. As Dash himself wrote in a more recent piece:

Applied CS over theory: A lot of yesterday’s computer science programs emphasized abstract concepts that could often be hard to translate into practical impact. Given that more students have access to technology in their everyday lives than ever before, recontextualizing CS education to connect directly to the tools and devices they already use can ensure that what we’re teaching is relevant. By analogy, we’re going to need a lot more electricians than electrical engineers, even if we know that the two related disciplines are both important and valuable.

The problem with this analogy is that nobody expects any electricians to become electrical engineers, just like no one expects some construction workers to become architects.

I think we want to try to build a tech job ladder that’s more like the restaurant industry. It’s not uncommon for a great, influential chef to have started their career washing dishes or peeling carrots and then worked their way up.

I’m not saying blue-collar coders should aspire to be a tech celebrity chef; people should be able to have a great career that affords them a middle-class life without needing a CS degree. But I do think we should build a pathway that’s more fluid and that encourages more mobility.

Writing Report on How to Truly Democratize Emerging Technology

I haven’t posted anything the last two weeks, and that’s because I’ve been swamped getting a new project off the ground. I’m going to take a number of lessons I learned from the Mixed Reality for All project and turn them into a report on what it will take to make robotics, AIA, mixed reality, and other emerging tech truly accessible to all communities. I’m hoping to finish it by the end of the year. In the meantime, I should be back to regular posting by next week.

The Robot/AI Jobs Debate Is Too Cramped

Using a small army of cheap, recent college graduates; 32 deep learning algorithms; and a Greek Oracle who just escaped from rehab, CrystalBall Associates has produced a forecast of the future they’re confident is 97.82% accurate. The good news: robots and AI won’t create mass unemployment. Instead, millions of people will have a “Last Mile automation” job, helping with training AI/robot algorithms and making the “edge case” decisions AI doesn’t yet know how to do. Their day will sound something like this:
“Yes, that’s a pair of black shoes.”
“No, you shouldn’t have taken that turn.”
“Yes, you should give a refund.”
“Yes.”
“No.”
“Yes.”
“No.”
“I’m not sure, ask my supervisor.”
“Yes.”
“No.”
“No.”
“Yes.”
Every day. For the rest of their working life.

While doing this mindnumbing, soul sucking work, they will live in a world where they are surrounded by a cornucopia of robots, AI, virtual and augmented reality, digital fabrication, and other dazzling, creative technologies.

Yay?

I spent the last six months immersed in the Mixed Reality for All project. Now that it’s over and I’m back to listening to the high-level debate over robots/AI, I’m struck by how cramped this debate feels.

Maybe it’s because of the experience I had teaching how to create virtual reality using A-Frame. I’ve been teaching coding for many years, and this is the first time I giggled so much while creating lessons. Typically, in the first class you learn how to get the computer to print your name and add two numbers together. With VR programming, you start by learning how to create something out of nothing: type a line or two of code and suddenly there are brightly colored balls floating around you. For a moment, I got a glimpse of what it could be like to live in a world where instead of playing a wizard, almost anyone could be a wizard.

We are about to have an opportunity that is unique in human history. It hasn’t gone unnoticed in the robot/AI debate, but it’s an afterthought. In between saying we don’t need to worry about jobs and arguing that we need more training opportunities for future “high skill” work and must do something about inequality, someone will toss in a line about how robot/AI will eliminate a lot of drudge work and create more room for creativity. But they never unpack the implications of that throwaway line.

I think it’s time we start.

To Make Coding Accessible, Fix Frameworks, Not Languages [Geeky]

I’ve started reading the research literature on how to make coding more accessible. One thing I’ve noticed is that almost all of it is focused on fixing programming languages. But given how coding has changed in the last decade or so, I’m wondering if focusing some of their efforts on libraries and/or frameworks would make more sense (WARNING: Very Geeky).

Take data science. Many say that the Python programming language has become increasingly important in data science, but that’s not really accurate. When people say “Python,” what they really mean are several libraries written in Python: tensorflow or PyTorch for machine learning/AI, pandas for slicing and dicing data, etc. And if you’re going to visualize your data in pandas, you need to master one of several Python data visualization libraries. Or you might use d3, a data visualization library written in JavaScript that’s responsible for a lot of the gorgeous visualizations you see on major news sites.

Even more importantly, having taken an intro class in Python doesn’t really prepare you to work in these libraries. Both pandas and d3 use an approach that’s called “functional programming,” and as I can attest from teaching them, they both look and feel wildly different than working in vanilla Python/JavaScript.

Similarly, the modern world of JavaScript web and application development — e.g., Node — has morphed into something that’s unrecognizable to someone who’d learned JavaScript many years ago. Even worse, there’s a remarkable amount of churn. If you’d taken the time to learn the Angular application framework a few years ago, by the time you’d really mastered it you’d be under a lot of pressure to switch to newer frameworks such as React or Vue.

And that’s a real problem for beginners: they can’t do a heck of a lot of useful work unless you’ve also mastered one or more libraries/frameworks. Let’s take one more example. Say you are a beginner who’s gotten excited about programming in AR/VR. To do it, you’d have to both learn a language and a framework — either C# and Unity or JavaScript and A-Frame (my favorite). Based on my limited experience learning/teaching them, wrapping your head around the frameworks, not the underlying language, is the biggest challenge for beginners. And as I’ve discussed previously, the stumbling blocks a beginner faces working in a framework may be very different than those for a language.

Finally, libraries/frameworks also offer one big advantage over languages for researchers who are trying to have an impact on the real world: as far as I can tell, most programmers switch libraries/frameworks more frequently than they do languages. Getting an ecosystem of Python coders to switch to a new language designed by researchers to be easier to learn? That’s a pretty tall order. Convincing them to switch from one Python data visualization library to another is a much easier sell. And in the JavaScript world? Folks have gotten so used to a ridiculous rate of framework/library churn that if a researcher created a new framework that was substantially easier to use, odds are good that a lot of coders would jump ship.

Obviously there’s still plenty of room to do interesting research on making languages easier to learn and use. But if researchers want to have a bigger impact, libraries/frameworks might be a better bet.

Are Orgs Like Uber Creating a New Workplace or Reviving an Old One?

I’ve been reading about the political economy of late 19th century US, and a quote from a recent article by historian Charles Postel caught my eye.

The advocates of the new work regime claim that it is paving the way to the “flexible” economy of the future. In the process, it is unleashing the inner entrepreneur of the Walmart “associate” and the Uber “peer.” But for millions of workers the new “flexibility” means little more than overwork and insecurity, and the advertised work regime of the future represents a throwback to the exploitative tyranny of the first machine age.

In the late 19th century, the telegraph, steam engines, and electric power changed everything. But they also changed nothing. Because, as before, most work involved long hours at low pay in domestic service, farm labor, construction, mining, and other strenuous jobs. Hiring was often day-by-day, and many workers operated as semi-independent contractors. The infamous “sweating system” meant families set their own hours, their own pace, in their own living spaces (tenements). Coal miners, too, often worked as their own bosses, getting paid by the ton. This “flexible” work regime translated into hazardous work, child labor, and physical and mental torment. Workers stood one accident or bout of unemployment or sickness from catastrophe.

Uber may be using an app to manage the people who do its work but the people infrastructure “stack” the app sits on is right out of the 1880s.

The tech world is skilled at making us believe that everything it’s selling is new. And because change is occurring at a blistering pace, it’s easy to assume that when we compare the present to the past it is “disrupting,” the only history we need to consider is the history of Right Before Now. It’s not.

The work arrangements of Right Before Now, where many people had one employer who provided steady employment and benefits, wasn’t natural or inevitable. People fought and bled to create it, to “disrupt” the previous people infrastructure stack so children could have a childhood and an unprecedented number of working families could benefit from the prosperity they helped create.

In short, let’s not get suckered by companies like Uber. Let’s stop talking about “disrupting” and start talking about what values we want the new world of work to uphold.

By 2030, the Global 1% Will Own Two Thirds of All Wealth

According to the Guardian newspaper, we’re likely headed towards an unparalleled concentration of global wealth.

An alarming projection produced by the [UK] House of Commons library suggests that if trends seen since the 2008 financial crash were to continue, then the top 1% will hold 64% of the world’s wealth by 2030

The reason for this disturbing prediction? Wealth was already far too concentrated before the financial crashes of 2008. And since then,

the wealth of the richest 1% has been growing at an average of 6% a year – much faster than the 3% growth in wealth of the remaining 99% of the world’s population

And as far as I can tell, this prediction doesn’t take into account the potential impact of robots/AI, which sure looks like it’s going to create massive increases in inequality if we don’t change the rules of the road.

If we want our democracy to survive the age of robots/AI, figuring out what to do about jobs isn’t enough; we also have to confront head-on the concentration of wealth.

Writing Our Stories, Writing Code: A Community-Centered Approach to Teaching Coding

While working on the Mixed Reality for All project, I started thinking about how you might teach coding to adults in low income communitities where instead of treating them as isolated individuals, you’d take advantage of the power of community. At one point I wrote a quick-and-dirty sketch of what such a class might look like. It’s pretty rough — for example, I had some ideas about how you could take advantage of existing instituions, such as churches or unions, but didnt have the time to think them through. But there are enough ideas here that I think it’s worth sharing.

Google has a VR development kit called Google Expeditions that lets teachers teach through stories by creating “immersive, virtual journeys” for their students. It’s pretty impressive. But when using Google Expeditions, teachers are limited to just the tools that Google decided to put in this toolbox. If teachers need more tools to tell their story, they’re out of luck.

What if people in low income communities could create their own stories in VR/AR, and instead of being limited to the tools in the toolbox a big corporation gave them, they could make new tools? Could they unleash their inner Grand Master Flash, who helped create Hip-Hop by morphing a tool for playing music — a turntable — into a tool for making music?

Or to put it another way: what if we can interweave the art of creating stories in AR/VR with learning  the craft of how to code?

It might look something like this:

In the first workshop, a group of adults meet for a Friday night and half of a Saturday. On Friday night, first they participate in a story circle. Then they learn how to use one simple coding technique to begin the journey of expressing their story in VR. For example, each participant comes up with 3 words that sums up their story. Using a code template that displays one word in VR, they create their first VR page that displays their 3 words (see example at the end). They finish by showing off their VR page and do a brief check-in about how they’re feeling about what they’ve learned so far.

On Saturday, they start adding a few tools to their storytelling toolbox — e.g., tools for adding a picture, a paragraph of text, some simple interactions and animation — so they can create the first version of their story. The workshop alternates between a little instruction, a lot of playing and experimenting with code — possibly working in pairs ala pair programming — and thinking together about storytelling & reflecting on their experience so far. Through play, they practice the same coding techniques over & over so coding starts to feel less scary and more like a means of expressing themselves.

The group meets again for a few shorter Saturday sessions that take place every other week. Each time they learn one or two more coding techniques, a little more about VR/AR design & how to tell a story, plus sharing their stories about the experience. In doing so, they also build the trust & community they need to help them get over any fears (which is at least 50% of the battle).  In between these sessions, they work on their own or with coding buddies on their coding skill and their story — and each week the main group doesn’t meet, there is an optional Saturday drop in session for anyone who needs a little help.

Then they start the 2nd part of the course: learning how to make tools to add to their toolkit. They begin with another Friday night – Saturday half day workshop, then meet every other week for shorter Saturday sessions. Although this part will be harder than Part 1, they will already have a solid grounding in some of the basic skills they need to do this, and they will have enough experience of thinking of coding not as this foreign thing that only super geeks can do but a way of expressing themselves. As a result, rather than each new technique being a chance to feel stupid, it’s a chance to expand how they can express themselves. And they will have a community of trust — a “Band of Brothers and Sisters” — to help them get through any parts that feel intimidating/scary.

In these sessions, they also begin to discuss what it might mean to make an economy where more and more people could make part of their living by making and sharing tools and other ways of creating value, wealth, ownership, and community in AR/VR. For example, what would it mean to say “we own what we make” in this new economy?

By the end of this part of their journey, those who want to continue should be able to start meeting on their own, getting help through the network they have already learned how to plug into of other folks around the country who’ve gone through a similar experience and who — with occasional help from world-class expert techies from around the globe —  have been helping to shape the path people take to keep improving their skills. And perhaps a few of them will learn how to help teach the next set of workshops.


Here’s what the augmented reality example above might look like:

Although you don’t have a virtual reality headset, you can still view and interact with these pages using a desktop web browser– either Google Chrome or Firefox. To move around, use the arrow keys to move forward and back and your mouse to rotate what you’re seeing. You can also view them on a smart phone, although on my iPhone I need to tilt it up before I can see the words.

Many years ago I taught adult education workshops on beginning HTML, and I’m pretty confident that anyone with a high school level literacy could be taught to be comfortable changing the one word template into their own 3 words and to understand what they’re doing. The key is a) starting with a group exercise that lets people understand what it means to code (my favorite: “the boss’s idiot nephew”)  and b) lots & lots of practice with supportive people.

Every Move You Make, Every Breath You Take, Big Tech Will Be Watching You

I think we can now confidently say we’re now at the point where Big Tech needs to be seriously regulated. Facebook is in the hot seat right today, but as New York Times research into Amazon and Google patents show, there’s plenty more to be nervous about.

A scene from a future Amazon hopes to bring about, as described in a November patent:

You’re taking to a friend on the phone or “within a detectable distance of [a] device” like Alexa. You say

The vacation was wonderful. I really enjoyed Orange County and the beaches. And the kids loved the San Diego zoo.

The device is running a “sniffer algorithm” that’s looking for “trigger words,” “often a verb indicating some level of desire or interest” such as like, love, enjoy, downloaded, hate, returned.

For each identified potential trigger word, the device can capture adjacent audio that can be analyzed, on the device or remotely, to attempt to determine one or more keywords associated with that trigger word.

So when you said “loves,” it triggers the device to analyze your conversation and extract the words “San Diego zoo.” It’d does the same thing for your friend, who replied,

When we went to Southern California, I fell in love with Santa Barbara. There were so many great wineries to visit.

So at the end of the call, your device could send you a discount for a San Diego season zoo pass and her a wine of the month discount. And all of the info about these keywords, including the context of the call, could be stored as part of a profile of you and your friend.

Google’s even more ambitious. They don’t just want to extract info from every conversation you have, they want every drop of info they can get. In their September 2016 patent about a home monitoring system, they envision the following, either in a homeowner’s— or renter’s— home could be closely monitored. According to the patent,

an audio signature matching a dining chair movement across a floor may suggest that an occupant is sitting in the chair (e.g., because the occupant may have presumably moved the chair to sit in it). Indeed, video inputs may confirm and/or identify that occupants are sitting in the chair and/or at the table. Additionally, smart device inputs may be used to obtain a number of contextual clues, such as utensil movement, conversation content, vapor detection, etc. For example, in one embodiment, the vapor sensors may detect the presence of food within the dining room zone, which may indicate that a meal is being consumed in the dining room….. an audio signature of keyboard clicking, a desk chair moving, and/or papers shuffling etc. may indicate that someone is working…. an audio signature and/or video signature may be associated with the sounds and/or images of teeth brushing in the zone 618. Next, additional characteristics may be determined (e.g., the sink being left on, a duration of teeth brushing, a speed of teeth brushing, etc.). These findings may me reported and/or recorded within the system (e.g., for subsequent control and/or reporting by the household policy manager [program ]) .

And all of this info could be sliced and diced by demographic info.

Demographic information may include, for example: occupant information such as: number of occupants, gender of occupants, age of occupants, ethnicity of occupants.

And the techniques for gathering this data could be pretty sophisticated.

By way of example, a video monitoring camera placed in the kitchen of the home can perform image processing on several days or weeks worth of captured data to determine how many different individuals it sees on a regular basis, to establish how many occupants live in the house.

What could be done with all this data? Well, the patent suggests, you could use it to identify children’s behavior.

For example, characteristics of audio signatures, such as speech patterns, pitch, etc. may be used to discern child occupancy. Next, the occupants may be monitored, specifically listening for low-level audio signatures (e.g., whispering or silence), while the occupants are active (e.g., moving or performing other actions). Based upon the detection of these low-level audio signatures combined with active monitored occupants, the system may infer that mischief (e.g., activities that should not be occurring) is occurring….. For example, it may be expected that certain activities be performed in quiet, thus indicating that the quiet activity is unlikely to be mischief. For example, reading a book, mediating, etc. are oftentimes performed in quiet…..

For example, audio monitoring, optical monitoring, infrared monitoring, etc. may be used to discern occupancy and undesirable activities of the occupants. In one embodiment, the contextual data may include audio signatures indicating “bully” keywords such as derogatory name-calling, elevated voices, etc. Accordingly, the system may monitor for and detect the use of such “bully” keywords. Additionally, in some embodiments, the contextual data may include audio signatures indicating the use of foul language.

And not just kids.

returning to the “I’ll be home by 5:00” assertion, the system may determine a current location of the household member and how long it would take to get from the current location to the occupant’s house (block 394 of FIG. 46). If the occupant’s house can be reached by 5:00, a determination is made as to whether the assertion/promise has been met (decision block 356 of FIG. 45 and decision block 400 of FIG. 46)…. If the assertion has been met, routine monitoring proceeds (block 358 of FIG. 45 and block 402 of FIG. 46)…. If the assertion/promise cannot be met (e.g., the occupant’s house cannot be reached by 5:00), a finding that the assertion cannot be met may be reported and/or recorded (block 360 of FIG. 45 and block 398 of FIG. 46).

What an abusive spouse — or an authoritarian regime — could do with this kind of automated micro surveillance is not a topic covered by the patent.

The system could also be used to monitor group behavior:

For example, if the household indicated a goal to spend more time doing activities together and the system monitoring indicates that the household is spending less time together or marginally more time together, the system may provide a reminder of the household goal to one or more members of the household (e.g., via an audible and/or visual alert in the household, via a text message provided to the user’s smartphone, etc.)…. If sufficient progress toward the goal is attained, a progress reward may be provided to one or more members of the household. For example, in the togetherness goal mentioned above, if the family spends 20 additional minutes together in a week, when the goal is to spend an additional hour together a week, a progress reward may be provided to the family.

And you could use it to see if you’re keeping up with the Joneses.

In one embodiment, household bragging rights may be a reward, by providing a neighborhood message to other participating households, stating that the household is progressing towards and/or has attained the particular goal.

These visions of a world in which Big Tech has the ability record any scrap of info about us aren’t from some random email an idiot marketing manager sent to their boss or videotaped off the cuff remarks at some conference. At least a few people at Amazon and Google thought they were nifty enough to apply for patents. That’s not remotely the same as official business plans. But the fact that there’s a corporate culture at places like Amazon and Google where these public legal documents would be considered OK should give us pause.

There are a lot of great, caring people who work at Big Tech companies who’d be appalled at the iPanOpticon sentiments behind these patents. But given the resources, power, and reach that Big Tech now has, we need to make damn sure that while they “move fast and break things” they don’t end destroying our liberty and our society’s future.