“That’s a mess of stuff right there,” said Mark Chiappetta, pointing to a bin piled two feet deep that looked like a haul from a manic spree through Walmart: bags of Tide Pods, a box of pencils, mascara sticks and lipstick tubes in their packaging, dish towels and even a couple of rubber ducks.

Over it loomed a robot, busy at work. As Chiappetta watched, it reached down into the bin, grabbed an item, lifted it out, dropped it into another container six feet away, then started the whole process again. The body of the robot was pretty standard — thick steel with an arm that can bend at what looks like an elbow joint. It’s the hand — or the gripper, as it’s known –– that’s peculiar. This particular gripper is the invention of Chiappetta’s firm, Soft Robotics: four, thick fingers made of bright blue inflatable rubber. As they opened and closed, jiggling slightly each time, they looked vaguely like octopus tentacles. They neatly grabbed a rubber duck by its head, then went back to nab a dish towel.

It’s unusual to make grippers out of inflatable rubber like this. Historically, robot grippers have been made of more rigid materials like metal or hard rubber because for decades, robots have mostly been used in heavy industry, like car and electronics assembly. Those traditional grippers aren’t great at picking up everyday objects, though, the way a human can. Chiappetta, Soft Robotics’s C.O.O., is part of a new generation of roboticists trying to fix that –– creating robots designed to be used for shipping and packing products in fields ranging from e-commerce to food preparation.

“We’re picking and packing all kinds of products, like frozen breads, produce, meats, all kinds of fish and raw proteins, skinned unfrozen chicken legs,” he explained, as we wandered around the Soft Robotics headquarters in Bedford, Mass., where his staff was designing and testing customized grippers for clients in late 2019. “Pretty much the most challenging items you can think of — items that are easy to damage by squeezing them too hard.” He walked over to a huge cardboard box and pulled out a 13-pound bag of laundry detergent, which, according to his customer, had proved beyond the abilities of a suction gripper; the bags would fly off as the arm swung around. "They're moving these at really high rates and trying to put them in boxes, and they're shearing off,” Chiappetta explained. His solution was to craft an enormous hand — roughly 15 inches long by 12 inches wide — with fingers mounted around the edges that could wrap around the bag, holding it snugly.

Covid-19, meanwhile, introduced a whole new array of challenges. A lot of companies were calling Chiappetta wanting to add robotic capacity to systems already in place, but there were entirely new queries as well. Meatpacking plants, for example, whose employees work in tightly packed indoor environments and have been very hard hit by the pandemic, were looking for ways to automate more of the process. Chiappetta is still trying to figure out what role Soft Robotics might play in this; cutting and deboning meat –– especially beef –– is sufficiently subtle and detailed work that robots can’t easily substitute for human labor.

Normally, designing a robotic system for a customer –– with a gripper customized for a specific task –– can take a year or more. But there was one pandemic-related request that Chiappetta and his team had a head start on. One of Soft Robotics’s customers is a European online clothing company whose business, Chiappetta told me, soared by 40 percent in the early months of the pandemic. While welcome, this caused a huge spike in items being returned. And processing returned clothing, it turns out, is a painstaking affair, requiring the sorting and scanning of flimsy pieces of fabric, sometimes in diaphanous garment bags.

Soft Robotics had been working with this company on the returns problem since 2019, but the pandemic created an urgency to get the system finished and shipped as soon as possible. So in July 2020, Soft Robotics delivered its first-ever gripper designed specifically to pick up and maneuver clothes: two small rubberized fingers that can extend out to pinch a garment, and six larger fingers mounted around the inner two that can provide a firmer grasp. Once the gripper has picked something up, a robotic arm swivels around and drops it onto a conveyor-belt system also designed by Soft Robotics, which whisks the item past an array of bar code scanners.

“Remember the Alien movies, where [the monster] had like a little inner mouth that came out with jaws, within the bigger jaws around it?” Chiappetta asked. “It's kind of like that.”

When the robot system is working at full capacity, Chiappetta said it can process returns twice as fast as a human. (He would not share the name of the European client because he said the company did not want its name in the press, where competitors could learn specifics about how it is automating.)

For anyone who has had a passive curiosity about robotics in the past few years, it may seem that picking up garment bags, lipstick tubes or sacks of detergent are not terribly impressive accomplishments. After all, the internet is rife with videos showing robots doing astonishingly complex things –– solving Rubik’s Cube, say, or doing front flips. But the truth is that these exhibitions tend to be one-off experiments produced in university or corporate labs –– exciting to look at and potentially useful down the line, but nowhere near ready for wide-scale use in the real world.

Yet robots have, in recent years, begun to develop some new abilities — less flashy, but of more real-world consequence. The true revolution in robotics isn’t in doing flips or solving puzzles, but in conquering seemingly mundane tasks like being able to pick up both a tube of lipstick and a towel — things that seem easy but have required a blizzard of high-tech innovation to achieve.

For more than half a century, robots have mostly been cordoned off in big manufacturing plants. They were powerful and immensely effective at performing single repeated tasks for years on end, but too expensive and dangerous for all but the largest companies to use.

That’s now rapidly changing. Thanks to new and cheaper technologies, new markets and new ideas, today’s roboticists are creating robots that are moving beyond the traditional factory floor into completely new industries –– packing goods for e-commerce, preparing food and working alongside humans in small mom-and-pop shops.

Cheaper, safer and more adaptable than ever, this new generation of robots is not only changing what can be automated, but who can automate, and how easily.

To understand the state of robotics today, it’s important to know where we’ve been. The dual impulses that have led to so much innovation in robotics have remained remarkably consistent throughout history. On one hand, there’s the imaginative, sometimes quixotic quest to emulate life — like the robots solving Rubik’s Cubes and doing front flips. On the other, there’s the more prosaic desire to mechanize tasks and solve problems. This second group accounts for the overwhelming majority of all robots that exist today.

The earliest robots were generally in the quixotic camp. Records of early Islamic societies tell stories of inventors who made lifelike, gear-driven robotic musicians that could play patterns on drums. A robotic monk made in the 1560s could independently move its arms and open and close its mouth in emulation of prayer. (It’s still functioning today, in the collection of the Smithsonian Institution.) By the 18th century, an engineer named Jacques de Vaucanson was making more complex entertainments — an autonomous flute player capable of breathing notes into an instrument and a duck that ate and subsequently excreted its food.

Every once in a while, these pursuits led to the creation of machines that were genuinely useful. This was the case with de Vaucanson, who, after successfully creating his robo-duck, zeroed in on a more pragmatic problem: the automation of weaving. This feat would ultimately usher in the first true wave of programmable labor.

Back then, weaving was incredibly slow and labor-intensive, requiring a “draw boy” to raise and lower a series of threads while a weaver passed a shuttle back and forth. De Vaucanson realized he could automate this by storing the pattern of the fabric as a series of spokes on a metal drum, much as a music box holds the pattern of music it plays. As the drum spun around, the spokes would trigger a set of hooks to raise and lower the threads — in effect automating the tedious work of the draw boy. It worked extremely well, but could hold only a comparatively simple pattern. Buyers of fabric wanted much more complex designs.

A half-century later, another Frenchman, Joseph-Marie Jacquard, made some innovations that dramatically improved on de Vaucanson’s loom. Most notably, Jacquard’s machine stored the pattern on a series of punch cards instead of a drum. Hundreds or even thousands of cards could be chained together, allowing the loom to produce complex, ornate patterns at a speed blazingly faster than humans. A weaver and a draw boy could produce only an inch of fabric a day. The Jacquard loom could produce two feet.

Jacquard’s invention went into commercial service in 1804, revolutionizing the production of fabric. It is generally considered the first successful industrial robot, in that it performed a complex series of actions automatically, following instructions set out by its programmer.

The next pivotal moment for robots didn’t arrive for another 150-plus years. Though there was rapid industrialization inside factories, it wasn’t until 1961 that the modern age of robotics kicked off with the invention of the Unimate, the first robotic arm. The automotive industry in the United States was rapidly expanding after World War II, and automakers were looking for any way to speed up their lines. Hydraulically powered with a rotatable, pincer-like gripper and capable of lifting 75 pounds, the Unimate was first deployed by General Motors to lift and stack hot, die-cut metal parts — a job that involved exposure to toxic fumes and was dangerous for humans. Soon, production rates doubled to 110 cars per hour as automakers worldwide organized factories around the massive robotic tools. By the mid-80s, a reported 8,500 Unimate arms were at work, more than half of them in the auto industry.

The Unimate proved that robot arms could transform an industry. The following decades became a race to see what new iterations of the arm could do. Messy, leaky hydraulic motors were replaced with electric ones, allowing for cleaner, smaller robots. Increasingly efficient algorithms allowed for ever more precise and rapid movements. And the innovations in arm technology came from all over the world: Japan gave us the precise, four-jointed Selective Compliance Assembly Robot Arm (SCARA); Switzerland, the nimble and speedy delta arm. Each new arm expanded the range of jobs robots could perform, from the crude stacking of car parts to the delicate tightening of screws into a radio or ultra-rapid placing of resistors on a microprocessor. In the second half of the 20th century, robots became a crucial element of mass global manufacturing.

By 1997, just over 30 years after the Unimate was introduced, about 650,000 industrial robots were at work in factories around the world. By 2019, that number had grown to over 2.7 million, according to the International Federation of Robotics, a nonprofit industry association. Industrial robots continue to make up the lion’s share of working robots today, deployed in the manufacturing of vehicles, computers, electrical equipment, machinery, rubber, plastic and chemicals. But they are no longer the primary driver of the robotics industry. The last decade has seen entire new industries emerge that are eager to automate, prompting new cycles of innovation and revealing new layers of challenges and opportunities.

The ability of robots to manipulate an object –– to grab it, pick it up, move it around –– is often compared, dismally, with the amazing facility of the human hand. That’s partly because our hands are so flexible; each one has 14 joints, or, in robotics terms, “27 degrees of freedom.” Human skin also has a useful softness, deforming gently around objects as we pick them up. Through the sense of touch, we can determine with exquisite precision how tightly we’re grabbing something, and vision allows us to pinpoint how far away an object is from our hands. Lastly, humans have common sense and semantic knowledge. We know basic facts about the world. We know that a pencil is going to be lighter than a brick, and also that while an empty shoebox is bigger than a brick, it is much lighter. We can look at a jumbled pile of household items, see the corner of a wallet poking out from underneath, and — from that tiny part — know that an entire wallet is in there, hidden. A remarkable bouquet of technical tricks goes into our ability to pick up a pen, a broom or a bicycle. No one fully understands how humans do it.

“There’s a whole bunch of stuff the human nervous system is doing, a whole bunch of stuff that we don’t even have a clue about,” said Rodney Brooks, a robotics pioneer famous for studying the difficult challenges of sensorimotor control and the co-founder of Rethink Robotics, which in 2012 released the influential two-armed Baxter robot.

In comparison, robot grippers have only very crude versions of these human capabilities. At the highest level — common-sense reasoning about how objects work — robots have nothing; artificial intelligence is not able to replicate the myriad abilities that allow humans to effortlessly handle millions of objects. And the actual manipulator — the fingers or hand that does the grabbing — was until very recently almost always some variation of the two-fingered pincer. It was made of metal (sometimes with rubber or plastic too) using an architecture that had been pretty consistent since the early days of the Unimate.

To be fair, that style hasn’t really needed to evolve. Traditional metal grippers have worked fine in large-scale manufacturing, where a robot often picks up the same part over and over. But just as factories making cars, radios and computers inspired roboticists of the last century, the explosion in e-commerce today has been inspiring them to create grippers that can pack up all our purchases and send them off. In 2019, e-commerce accounted for 11 percent of U.S. retail sales, up from 9.9 percent in 2018, according to the U.S. Census Bureau. In 2020, the pandemic caused online shopping to rise by fully 31.8 percent between the first and second quarter of the year. And though large e-commerce and shipping companies like Amazon and FedEx are highly (but far from fully) automated, many smaller companies are not. Accurate numbers on this are hard to come by, but DHL Trend Research estimated that 80 percent of shipping work –– lifting, grabbing and packing –– was done by people in 2016. More recently, MHI, a non-profit trade group, published a report surveying industries involved in a much larger sphere of work –– material handling, shipping and supply chains. In 2020, only 39 percent of respondents claimed to be currently deploying robotics and automation, while 73 percent expected to be using the technologies within five years. And although the long-term effects of Covid-19 on consumer behavior won’t be known for a while, it seems clear that the huge surge in e-commerce demand over the past year will accelerate online shopping habits. This will prompt more and diverse retailers to hunt down tools that will allow them to cultivate and streamline online sales.

One result of all this has been a rush by roboticists to replicate some of the magic contained in the human hand. One such roboticist has been George Whitesides, a chemistry professor at Harvard who co-founded Soft Robotics and pioneered the rubbery fingers. His breakthrough came after years of puzzling over the challenge of manipulation. One of things that historically made manipulation so hard, he observed, is its computational intensity. The simple act of efficiently grabbing an object requires visual recognition (assessing where the object is), planning (figuring out how to reach the object) and feedback sensing (how firmly to grasp the object).

“To pick up a strawberry,” Whitesides explained, a traditional gripper would need an enormous amount of data about the strawberry’s location and firmness. “You have to have sensors and that information has to be fed to the computer.”

But he realized that when humans grab objects, some of the hard work isn’t done by our brains. It’s done by the physical properties of our bodies — the flexibility of our limbs, the softness of our skin. This, he argues, is part of what makes human fingers so powerful. Our sensors — touch and vision — are remarkably accurate, but our fingers are also quite forgiving: They deform around objects we pick up, and thus compensate for any errors made in visually figuring out the location of the object. So he wondered: “How about having the material be the controller?” Some of the required computation could be transferred to the material of the gripper, as it were.

Indeed, he also suspected that human hands weren’t necessarily the best grippers to emulate; the octopus, with its exquisitely prehensile tentacles, would be even better. In 2011, Whitesides began publishing the results of experiments on the value of softness in gripping. In 2013, he co-founded Soft Robotics to take his products to market.

One of the company’s first hires was Chiappetta, who had worked on grippers and manipulators with Joseph Engelberger, the businessman behind the commercial expansion of the first industrial arms in the ’60s. Chiappetta had also created and designed robots at iRobot, the firm that makes the Roomba, and was enchanted by Whitesides’s concept of getting human performance without having to mimic a human form.

“His approach was biological inspiration, but it was not human biology,” Chiappetta told me when I was visiting his lab.

The rubbery fingers are paired with a vision system to help the robot arm recognize objects, but the robot doesn’t have to be precise in its reach; if the arm gets the gripper roughly in the right place, odds are high it’ll grab the item. “We literally don't have to worry about the shape of the item," Chiappetta explained. “We don't have to worry about where to optimally place the fingers; we don't have to worry about how much force to apply. Because the fingers themselves interact with the items and they form an ‘enveloping grasp’ of sorts.”

That makes the system far more adaptable to new uses, because it can handle different objects without specific training. It also reduces the extent of 3-D planning that the robot system might otherwise need to engage in to get the gripper in the right position. Shaving seconds, even milliseconds, matters. The less processing the robot requires in order to grab something, the faster it can do its work.

“These are going out to e-commerce companies,” Chiappetta says. The orders came in November, from firms needing them in operation in their distribution centers by April 1. "We're running,” he said with a laugh. “We're all working really late and weekends to build them.” (He can’t tell me which companies ordered them; most of his customers want to keep their use of soft robots a trade secret from their competition, he says.) He has 50 people building his systems now and is still hiring; he imagines eventually supplying manipulators to work in everything from industrial assembly to military projects and maritime work. For now, logistics and packaging are giving him as much work as he can handle.

Soft Robotics is of course not the only company that has been working on an über-gripper. It’s not even the only one in the Boston area. The opportunity to create a universal gripper for the variety of machines demanded by the growth of e-commerce has created a gripper gold rush of sorts. Over at MIT, Daniela Rus, the director of the Computer Science and Artificial Intelligence Laboratory, has been developing a soft, cone-shaped, vacuum-powered gripper that deforms around and picks up cylindrical objects. And in nearby Somerville, Mass., another start-up called RightHand Robotics is tackling the challenge from multiple angles: jointed fingers, suction and a data-driven visual recognition system.

I met RightHand’s co-founder and C.E.O., Yaro Tenzer, at a trade show in 2019, where he was demonstrating his gripper. Mounted on an arm made by Universal Robotics, it was picking up an array of objects from within a plastic tote and failing only very occasionally. They ranged from cosmetics compacts to sponges and, at one point, my wallet. “Try it out,” Tenzer said. I dropped it in the bin, and the robot successfully nabbed it on the first try.

Part of what makes the system flexible, Tenzer explained, is the combination approach: The suction device is able to latch onto most objects it would typically encounter in a shipping warehouse — a box, a tube of toothpaste, a rubber ball — and pull it up into the grasp of the three fingers. It can also work without the suction, automatically switching to its fingers to grasp an object directly. For its vision system, RightHand uses the Intel RealSense camera and an algorithm that enables the robot arm to pick up items it has never seen before. The company also networks every robot it sells, so if the robotic gripper encounters a previously unknown object and successfully picks it up (or doesn’t), the data goes back to the cloud. At this point, according to RightHand executives, the vision system has encountered many more than 10 million successful “picks.”

Is it likely that there will be a single winner among the manipulation and warehousing bots? I suspect not. The various grippers overlap a lot in their capabilities, but each is subtly different in ways that might make it more — or less — desirable for certain tasks. The sheer range of possible tasks to automate is so broad that the range of gripper companies will probably continue to grow for years to come.

Not all improvements in manipulation are coming from grippers, and no single innovation is likely to vault the robotics field forward. Indeed, robotics is not really a single field. It’s several interlocking disciplines: Materials scientists work on gripper materials; electrical engineers work on the circuitry and motors; mathematicians develop algorithms to help robots plan the way they’ll move their arms from point A to point B. A robotics company requires an “Ocean’s Eleven” team of expertise, and a breakthrough in any single domain can improve a robot’s abilities.

One area in particular, however, has dramatically expanded the capabilities of robots in recent years while also making them more affordable: vision. New types of 3-D cameras and a renaissance in artificial intelligence have allowed today’s roboticists to deploy remarkably powerful new ways of seeing and perceiving the world for a fraction of what it once cost.

Until the last decade, creating a vision system was a painstaking and often labor-intensive process with relatively crude results. Computer vision programmers would write software specific to a single object the robot needed to identify — say, a mechanical part going by on a conveyor belt — by defining collections of pixels representing crucial features of that object, such as its corners or its color. Then, when a camera delivered a stream of pixels to the robot, the robot would respond to the pattern it had been programmed to identify. The method worked, but it was tedious, required an enormous amount of programming labor and only enabled a robot to recognize objects it had been explicitly trained to look for. If you wanted it to “see” a different object, you had to start from scratch. The sheer amount of programming required to create a vision system this way meant that it was used mainly by established and well-funded companies.

But two significant breakthroughs have made computer vision much easier and cheaper to deploy, while delivering much higher quality.

The first was in camera technology itself and came from an unexpected source — the Xbox and its Kinect motion-sensing technology, which contained an RGB camera similar to the ones used in some mobile phones, with a 3-D-sensing chip and an infrared sensor. The system could shoot infrared light out at the scene around it, allowing the sensor to measure how quickly the light bounced back from nearby objects in what’s known as “time of flight” sensing. Merged with the Kinect’s software, this data allowed the Kinect to visualize objects as a point cloud: a cluster of tiny dots outlining an object in three dimensions. (Imagine your body covered in thousands of tiny dots; that’s how a Kinect “sees” you.) As Microsoft began selling millions of Kinect devices, it didn’t take long for roboticists to realize that this style of sensing would be enormously useful for robots. Similar 3-D devices were entering the market from other firms, such as the Intel RealSense camera in 2015.

The second breakthrough has been the advent of deep learning. While improved cameras certainly helped robots receive better, more detailed data about the world around them, they still needed software to help them identify what they were seeing. A 2012 breakthrough in deep learning enabled computers to learn and recognize objects by finding patterns in vast amounts of data, transforming a robot’s ability to perceive objects. Thus, in the span of just a few years, image recognition went from being a bespoke and time-intensive feature that only the most well-funded companies could afford for their products, to one that even small start-ups could use, thanks to access to free, open-source code.

All these advances in robotics — better grippers, cheaper arms and powerful, affordable vision systems — have allowed smaller businesses to introduce robots at much lower prices. Whereas a big industrial robot might once have cost in the hundreds of thousands of dollars to buy, program and install according to a 2007 estimate from the industrial supplier ABB, these newer innovations have helped produce robots with total costs as low as $50,000, making dexterous automation affordable to many more buyers.

One such buyer is Tommy Caughey Jr., the C.E.O. of WALT Machine, Inc. Since 1995, WALT has been making precision-machined parts, with a specialty in crafting assemblies for scientific cameras and military hardware. For years, Caughey had been thinking about automating some of his labor, but believed a robot would be too unadaptable and hard to program. “I thought it’d be really cool to have one, but I thought it was a yellow robot in a cage and it was only good for doing the same part over and over,” he said. WALT’s business depended on being able to take on a variety of different jobs, meaning the company would have to reconfigure its robot frequently and quickly. He had only two full-time employees and neither was a tech specialist. Who was going to program the thing?

Then, at an automation conference in 2016, he saw a setup made by a company called Robotiq that might fit the bill: Attached to the end of a robotic arm was a very versatile and flexible gripper with a camera. It was simple enough for a layperson to use and program on the fly for different tasks.

If the fulfillment industry represents one new sea of opportunity for robots, small and medium-size businesses represent another. Such companies account for 98 percent of all manufacturing firms in the U.S., and three-quarters of those firms employ fewer than 20 people.

For reasons of cost and scale, those small businesses have been far less automated than their larger industrial counterparts. But small-scale manufacturers like WALT are an emerging market for robotics companies like Robotiq. “Tiny companies,” is the way the Robotiq co-founder Samuel Bouchard describes many of his customers.

In 2008, Bouchard and two co-founders set out to make manipulators that small firms like WALT could quickly put to use to help automate processes like assembly, picking and placing, and machine-tending (a repetitive and sometimes dangerous job that entails loading and unloading raw materials into machines to be shaped). “The demand for robots is increasing faster than engineers are getting out of school,” Bouchard explained, “so you not only need robots that are more capable, but also robots that understand more seamlessly what the human wants to show them.”

To achieve this, Bouchard and his team focused not on a new gripper design but on a vision system, ultimately designing a 2-D camera system mounted onto the wrist of their grippers, coupled with some extremely easy-to-use software to teach the robot. Users position the camera so that it’s looking down at whatever the robot needs to pick up. The arm takes 13 pictures of the object from several angles, and in a few minutes is able to recognize a new object and pick it up. To speed up the training, a user can also use the system’s touch screen to draw a box around the object they want the robot to recognize. The process is not lightning-fast, but the system’s value is not in its speed. Its advantage is in how easy it is to configure and reconfigure. If you suddenly need the robot to pick up a new, different object, it can be retrained in short order.

Caughey said that the robot doubled his firm’s output. After installing it, he started running two shifts, adding an evening shift when the workload was heavy. The robot feeds metal pieces into three computer-controlled lathes, while a manager oversees the work. This ability to run longer hours became particularly critical when Caughey won military contracts for jobs that were more than an order of magnitude bigger than anything he’d ever done before.

“It was two massive jobs that ran back-to-back — one of them was for maybe 60,000 parts, and the other one was for over 100,000,” he said when we first spoke in 2019. “To put it in perspective, we’ve made more parts in the last 12 months than we made in the last 10 or 12 years.”

His robot system cost $55,000 total: $45,000 for the arm, $5,000 for the camera and $5,000 for the gripper. Caughey said he hasn’t precisely calculated how quickly the investment paid for itself, but estimates it was around 250 days, mostly by expanding capacity and allowing him to accept jobs he once had to turn away. “We can now do large production jobs,” he said. Also, he has more than doubled his staff, expanding to six employees, five full time and one part time.

“We need new types of people,” he said. “Before the robot, we wouldn’t be running two shifts. Now we have to have two shift supervisors. So you need a higher level of skill.” He also hires fewer temporary workers, and he can offer people more interesting work. One machinist who’d previously been doing machine-tending started retraining in new skills, including programming the robot and inspecting parts. It is extremely hard to find labor, Caughey said. To pull off a job the size of the 2018 military contract without the robot, he would have needed up to 10 people — “and I don’t even know where I would find 10 people, especially ones trained in manufacturing. It really is an art.”

Cheaper vision technology — particularly the plummeting cost of Lidar (an acronym for light detection and ranging) — is transforming warehouse work as well. The cost of a Lidar system, a form of laser vision most famously used in self-driving cars, has dropped from $75,000 in 2007 (for a system from Velodyne, a pioneer in the field) to as low as several hundred dollars (a system suitable for a small mobile robot). That has contributed to a recent surge in the development of autonomous mobile robots (AMRs), the rolling carriers used to ferry goods around warehouses. Kiva Systems was the first to bring an AMR to market in the late 2000s, roughly doubling the rate at which items could be picked for packing at firms like Walgreens. When Kiva was purchased by Amazon in 2012, competitors like Fetch, a company based in San Jose, Calif., swooped in to fill the void.

Like Robotiq, Fetch has aimed its products not at mega-companies but at smaller operations without a lot of technological infrastructure. A single Amazon-style mobile delivery robot might cost about $40,000, but that would not include programming costs. Fetch, and firms like it, are aiming to eliminate or reduce such costs by creating machines that can be programmed by neophytes. “We looked at our worst-case user, which is someone who has no tech background and finds computers difficult to use,” the C.E.O. and co-founder Melonee Wise told me. The result is a growing array of accessible systems that allow small- to mid-level players in the shipping and logistics industry to automate in the style of an Amazon.

What sorts of efficiency gains, or productivity boosts, will the rapid expansion of service robots deliver inside warehouses? Again, it’s not easy to make firm predictions. But some case studies are instructive. John Santagate, now vice president of robotics at HighJump Software, a warehouse supply management firm, did a study of robotics systems for the International Data Corporation. He examined the effect of these new warehouse bots on a company called ACT Fulfillment in California, which does warehousing and shipping for fashion brands. When ACT discovered that its staff couldn’t manage increased demand fast enough, it ordered 10 mobile robots from 6 River Systems. Those robots tripled the rate at which products were picked and reduced mis-picks — the wrong item being selected — by 90 percent. The amount of time that humans spent walking around the warehouse finding items was reduced by 50 percent. The upshot is that a seven-person team processed orders three times as quickly. ACT Fulfillment figures it broke even on its robotic investment in only five months.

Fetch and 6 River Systems, along with other firms, are enabling small companies to lease AMRs along with access to services like maintenance, upgrades, I.T. support and the option to swap out robots when needed. So instead of buying an AMR or an arm with a gripper for $50,000, companies can now essentially get robotics on tap, paying as they go. Firms like Universal Robots have launched similar leasing plans for so-called cobot arms, which work alongside human employees.

“Instead of being a capital expense,” Santagate said, “it becomes an operating expense. In theory, in certain scenarios users can get a zero-day ROI.”

What’s coming into focus is a world of tighter collaboration between robots and people. It’s the rise of “cobots,” as the industry calls them, where the baseball-capped, bearded machinists at WALT toil side-by-side with a swinging, gray-and-blue robot arm.

The idea of robots and humans working literally next to each other is a significant development in automation, as historically robots have been too dangerous while operating for humans to get near. But in the early 2000s, a roboticist named Esben Østergaard got the idea that there was a large latent market for smaller-scale, lighter-weight robotic arms that would be less dangerous and easier to use.

In 2005 he founded Universal Robots, which sold its first robot in 2008. Over the next decade the Universal arm became ubiquitous in smaller firms worldwide, with over 30,000 sold to date. (Østergaard left the company in March 2019.) Many are used — as at WALT — to feed materials into and retrieve parts from shaping machines, while others perform tasks like drilling holes for construction, helping rebuild the facades of historic buildings or even packing eggs into shipping boxes.

“It’s about changing who can automate, where can you automate, and how much trouble is it to automate,” Østergaard said.

But while robotic arms have gotten smaller and — depending on what they’ve been assigned to do — less dangerous, the safety challenges aren’t considered solved. Even if a cobot arm is lightweight enough that colliding with it wouldn’t cause injury, said Jeff Burnstein, president of the Association for Advancing Automation, “what if the arm is carrying a knife?” Large manufacturers do extensive risk assessments before they deploy heavy robots, which is part of the reason there have been only 43 incidents of worker injury or death by industrial robots since 1984, according to the Occupational Safety and Health Administration. As cobots emerge in smaller firms, safety will be an important factor in widespread adoption.

Is it possible that all robots — even huge industrial ones —could be made safe? This has been the question driving another start-up, Veo Robotics of Waltham, Mass. The company’s goal is to create a vision system that can be used with existing industrial robots to make them compatible with human co-workers.

The company was founded by Patrick Sobalvarro, a robotics veteran who, in the late 2000s, was the president of Rethink Robotics. He loved the idea of collaboration between human and robot, but also knew that big industry couldn’t use the safer lightweight arms — they needed strong steel ones. “They love the robots they already have,” he said, “they just need the robots to be more adaptable.” Auto manufacturing still relies on many humans; only people, thus far, have the dexterity and reasoning to quickly adapt to new customization needs. Still, automakers and other manufacturers could speed up a lot if the humans could work more closely with the robots, each doing what they do best.

“For both economic efficiency reasons and ergonomic reasons, what you'd really like to do is have that robot do the heavy lifting, and place things for assembly — present them to people at the right angle and so on,” he explained. “And that lets people concentrate on the things that really only people can do, like use their dexterity, their judgment, their quality-control abilities.” It wasn’t a new idea, as Sobalvarro noted. For years, “people have been saying, ‘Yeah, it'd be great to have robots be aware of the presence of humans.’”

But it’s harder than it sounds, because of the occlusion problem: If a robot picks up a car door and a human crouches behind the door, the vision system might not see the human, and therefore think the worker isn’t there. What is needed, Sobalvarro realized, is to blanket the whole work area with 3-D vision and also track the occluded area. But when he was originally pondering this problem, 3-D vision wasn’t cheap or fast enough. To be truly safe, you’d want a vision system scanning the work area every 30th of a second or so. Nothing could yet work so quickly.

By 2015, though, technology had improved. The second generation of Kinect-style sensing had come out with faster chips. By 2016, Sobalvarro decided the camera tech was good enough to start working on the problem. He set up Veo Robotics and hired a team of engineers and vision experts to start experimenting.

A critical early decision was not to create a neural network to identify objects. Neural networks are great if it’s OK to get things right only most of the time. But they don’t guarantee 100 percent accuracy 100 percent of the time. And Sobalvarro, given the size of the machinery his team is working with, wanted that level of reliability. “It's kind of hilarious when Google's algorithms identify a bicycle as an elephant,” Sobalvarro said. “If that happens one in 10,000 times, Google declares victory.” But with 12 million people working in factories every day, “one mistake in 10,000 is not good enough. You're gonna get somebody hurt.” What’s more, a neural network cannot yet engage in the logical, rule-based thinking necessary to help figure out if a work area is safe: If I saw a human in that spot a second ago and now there’s a door shape, is the human simply hidden behind the door?

So he and his team at Veo set up a system that includes four to eight ceiling-mounted 3-D-imaging cameras that feed data into a computer system monitoring a designated area. The goal, Sobalvarro said, was “to prove that a human can't be in that occluded area.” If a human gets too close to the robot, the system rapidly slows the robot down and eventually says, “Stop everything.”

When I visited Veo’s lab, I saw a six-foot-tall orange Kuka robot whipping around in a circle, lifting up thick metal parts and placing them on a table. In normal circumstances, a robot this size would be caged. Here, it was operating within feet of Gwenn Ellerby, then leader of Veo’s design assurance team, who was tightening bolts on parts delivered by the arm. This put her head directly in the path of the Kuka arm. But whenever she leaned forward, the robot stopped moving. When she moved out of the arm’s range, the robot resumed what it was doing. At several points Ellerby intentionally walked toward or reached up to the robot while it was getting a new part, instantly causing it to pause.

“Come on over here,” said Clara Vu, Veo’s co-founder, pulling me over to a large screen showing how the vision system was processing the scene. Each mass in the room was represented by a cloud of tiny points in different colors. “The red? That’s Gwenn there,” she pointed out. The robot itself was in the middle of a huge bloom of yellow dots. “Those represent the future robot cloud, all the places the arm could get to,” Vu said. Veo’s system is constantly computing the distance between the robot’s current position and any potential obstacle, to figure out how soon it needs to slow down to prevent collision.

Vu pointed up at the four cameras looking down in each corner of the work area. Back then, their prototype was — remarkably —using four Microsoft Kinect 2 motion sensors, the same ones you’d find on an Xbox. They functioned well enough for this prototype, but Vu and her team have since developed a custom 3-D camera that includes multiple layers of safety systems.

Sobalvarro and his team received safety certification for the system in April 2021 from TÜV Rheinland, an industrial manufacturing regulator based in Europe, and are now preparing to roll it out to manufacturing customers in the US, with Europe to come later this year. The customer base he’s working with consists of manufacturers in several industries, including automaking (where fully 30 percent of all industrial robots are used), major appliance makers, systems integrators and a few large aviation firms. By the middle of 2020 customers were testing systems; after Veo gets its certifications, the systems can be rolled out onto actual production lines. At the moment, he tells me, most of his customers are looking to retrofit existing robotic cells. Covid-19 has become, he said, another issue that some customers raise when they talk to him about how they could use Veo’s system. “We’ve heard people say, ‘You know, I want that heavy part moved with the robot because I can't put these people that close together,’” he noted.

Sobalvarro imagines a workplace where, eventually, a human can work alongside big, swinging, heavy arms without risk of injury. It would be, fancifully, a bit like Moses walking into the Red Sea: "The robots are aware of the people and they're there to serve the people, and they do what is necessary to allow people to move through there safely.”

If the first wave of robots announced itself in the workplace with the shuddering thud of steel on a concrete floor, this wave is arriving more like a steady and increasing wind, adapting itself to multiple needs and spaces and taking different shapes as it does so. For this reason, the adoption of these new robots by companies — at least so far — is harder to quantify. The International Federation of Robotics (IFR), which has been tracking the sales of robots and their markets at least since 1992, started accounting for nonindustrial robots in their tallies only in 2009. Many of these new robots fall into the category known as “professional service robots,” meaning that are not directly involved in the manufacturing or assembly of products. Instead, they perform a service — like milking cows, facilitating medical procedures or sorting and ferrying e-commerce products.

As the IFR noted in a recent report, sales numbers for these professional service robots are harder to track and possibly less accurate than those for industrial robots, because both the makers and users of these new robots are far more fragmented than, say, those involved in the production of cars. With that caveat in mind, data produced by the IFR shows industrial robots still accounting for the majority of unit sales of robots used in the workplace today, growing from over 60,000 sold globally in 2009 to over 370,000 in 2019. Sales of professional service robots, meanwhile, are growing speedily alongside them, going from over 13,000 to over 173,000 in the same time period. And while trade tensions and economic softening in 2019 prompted sales of industrial robots to dip, according to the IFR, the market for professional service robots continued to rise.

What seems clear is that this generation of robots will spread automation to places it has never been before and put it into the hands of business owners who are making decisions about robots for the first time. That includes the roughly 187,000 manufacturing companies with under 20 employees (55 percent of which employ four people or fewer). For them the range of tasks that could be automated is almost infinite, from the machine-tending at WALT to the lifting of heavy bags of detergent by Soft Robotics grippers. As robots continue to become cheaper and more adaptable, it’s inevitable that the number of Tommy Caugheys will grow.

Caughey, indeed, was inspired enough by the productivity gains at his own factory that he now offers consulting to other small and mid-size factories about how to automate. He purchased a second robot in 2020, and in speaking to clients over the last year he’s seen Covid-19 become a spur for automation. “It’s put a strain on manufacturing for everyone,” he said, “and it's not just small shops, it's across the board.” One company he met with in the fall of 2020 mixes dry ingredients for products like chicken seasoning, and “they just can't get people to come to work,” he said. So they were thinking of investing in robots to pack the bags of product into boxes for shipping.

The pandemic has created a unique pinch for many firms: increased demand coupled with labor challenges. Yaro Tenzer, the C.E.O. of RightHand Robotics, said that as the pandemic progressed, he fielded many more inquiries from shipping and logistics firms looking to add robotics to deal with increased demand, because it was too dangerous to add more labor. “There are companies reaching out and saying, ‘Hey, before we could just throw labor at this –– at the moment we cannot,’” he said. “Before, you literally put like twenty people back-to-back to do things.”

How much of an impact might automation have on smaller-size firms? Given the lack of accumulated data about the applications of new machines, projections of impact are a gamble, but a 2015 study by the Boston Consulting Group estimated a global output-per-worker productivity increase of up to 30 percent before 2025, driven in part, the report stated, by a newfound accessibility to robots for smaller manufacturers.

The proliferation of robots in smaller businesses will no doubt have a profound effect on the work force, though exactly what that will look like and how quickly such changes will evolve remains to be seen. These are important, significant questions that go well beyond the scope of this article. In brief, however, there is no clear consensus among economists on whether the coming waves of automation will create enough new jobs to replace those lost to robots, and whether the new jobs will be as good as those lost to automation. It’s also not clear how uniformly or rapidly these new robots will spread in different regions and industries; local economies and labor markets vary, as does the decision-making of every individual firm.

Take a look at the growing field of third-party logistics, or 3PL, which includes the universe of firms that manage all the sorting, packaging and ferrying required to deliver e-commerce products to customers. Even before the Covid-19 surge, 3PL grew 21.6 percent between 2016 and 2018, largely led by increased automation in large multinational firms like DHL, UPS and FedEx.

But even in this ripe-for-automation industry, not every company chooses to automate. Jay Catlin is C.E.O. of AMS Fulfillment in Santa Clarita, Calif., where 150 workers pick shoes, lipsticks and shirts from shelves, package them and send them out for shipment at a rate of up to 600 items per labor hour depending on the order metrics, and far beyond that for business-to-business orders. No part of his company is automated, and the pandemic has not made him a sudden robotics convert, despite a surge in demand.

"The pandemic has been very impactful,” he said. “We've had to put tons of expensive safeguards in place. We've had to do more business and do it with revised processes and protocols. It's been very challenging. We're always looking for ways to improve efficiency, but that doesn't for us mean necessarily reducing headcount."

AMS has about $100 million in revenue, so Catlin could afford a robotics system, but investing in a new technology that may disrupt business is a riskier bet than sticking with human labor, he said.

Across the U.S., one thing that’s holding some companies back from automating more rapidly is that labor is still comparatively cheap. In 3PL, freight volume and value have increased over time, but average annual pay hasn’t. According to a 2019 report by the Berkeley Labor Center on the future of warehouse work, wages are stagnating in the sector, falling slightly in adjusted dollars from a peak in 2001 of just above $40,000. The authors of the report concluded that automation will likely lead to a slow reduction in the importance of warehouse workers, rather than sudden and massive job losses. Still, that slow reduction will have downsides, they argued; workers might keep their jobs, but they’ll face "wage stagnation and job insecurity."

Cost, however, is only one reason Catlin still prefers human employees (whom he recruits from places like prison work-release programs). While he’s open to automating, so far he hasn’t seen a robot with the agility necessary to manage his constantly changing inventory and customer demands, which include packing tubes of mascara and placing shirts in plastic poly bags. “Say that we have a seven-foot-tall Barney plush toy that needs to be packaged,” he said. “I don’t see how that is automatable.”

Still, for other business owners, investing in robotics is an essential bet on where the future is going. “Do you think in 200 years that I would be running pick-and-pack manual operations?” asked Thom Campbell, co-founder of Capacity LLC. “No way.” Capacity, which manages e-commerce fulfillment for cosmetic companies and ships to retailers like Sephora, tapped RightHand Robotics a few years ago to help it replace one of the more mundane tasks at its warehouses — separating batches of items that have already been picked from the inventory and putting orders together at the “put wall” or sorting station.

Humans at the put wall can place 400 or 500 orders per hour, which is actually about 50 percent more productive than a robot in the same time frame, Campbell said. But a robot works overtime at no cost. In two shifts a robot can reach the same productivity as a human, while exceeding it by the third shift, Campbell said.

Like many in the fulfillment industry, Capacity saw demand surge during the pandemic, with many months of 2020 setting records. The company was fortunate to have invested heavily in automation pre-pandemic, but that there is still only so much of the process that machines can do. “We’ve tried to remove as many human touches as possible,” Campbell explained, “but there’s still an enormous number of human touches required. I’m talking to you and looking at a warehouse full of people.”

To keep up with demand, the company needed healthy employees. So early on in the pandemic, Campbell said, large quantities of masks and hand sanitizer were procured, cleaning was supercharged, employees were physically separated as much as possible and segregated into groups that stayed together throughout the day. Hazard pay was given and then those raises were locked in. While the pandemic has meant more business for Capacity, it has also meant an extra $2 million in labor and over a quarter-million dollars in cleaning costs. Speaking in November of 2020 and preparing for Black Friday and Cyber Monday, it was more people, not machines, Campbell wanted. But increasing density and bringing new people in –– some perhaps just temporarily –– did not seem feasible given Covid-19.

Over time, he knows, more automation will creep in, task by task. Innovation in robotics was extraordinary in the second half of the 20th century, Campbell said, but it has been meteoric in the last 20 years. He said he believes it will be a necessary investment not only for Amazon and DHL but also for other smaller players like his company. “It’s the Amazons and Walmarts who have the incentive to innovate and the extraordinary access to capital,” Campbell said. “And if you can’t compete with the expectations they are setting, you’re in trouble.”