I saw the future of AI in Windows at Microsoft Build 2024, and it’s exciting.
I’m talking specifically about the new features for Copilot+ PCs that Microsoft made a big show of talking up at Build, most notably Recall and Cocreator.
At least some of these features have been available to Windows 11 Insiders in some form for some time now, but according to Microsoft they’ll be exclusively available on Copilot+ PCs when devices start hitting store shelves June 18.
If you missed it, a Copilot+ PC is a new marketing term from Microsoft that basically means a Windows laptop or desktop with at least 16GB of RAM, 256GB of storage and an NPU (Neural Processing Unit) capable of at least 40 TOPS (trillion operations per second) onboard.
Microsoft’s own Surface Pro 11 and Surface Laptop 7 will be some of the first Copilot+ PCs thanks to the Snapdragon X Plus and Snapdragon X Elite chips inside, but Acer, Dell, Samsung and other laptop makers are also shipping Copilot+ PCs in June.
Select features of Windows 11 will only be available on Copilot+ PCs, perhaps because they require at least a 40 TOPS NPU or perhaps because Microsoft wants to give you a reason to buy a new PC. Whatever the reason, these features are worth knowing about if you’re thinking about buying a new PC this year because they show you what the cutting edge of Windows AI can do.
While I was at Microsoft HQ for Build 2024 this week I had a chance to see demos of some of the new Windows features coming only to Copilot+ PCs this summer, and there are a few that caught my eye. Here’s a quick rundown of what they are, how they work and how they could change the way you use Windows.
Recall
Recall is the big new headline feature that’s going to be available on Copilot+ PCs, and it’s a pretty significant addition.
When available it sits in your system tray and basically works like an expanded search function for your PC, but it can search not just through data on your PC but also the history of what you’ve done on it. So if you were to search for an image of a boat or a webpage about boats you were looking at last week, you could type “boat” and Recall will show you a scrollable timeline of everything you’ve looked at onscreen over the past month or so that includes images of boats or the word boat.
That’s a very simple explanation of what appears to be a complex new system which captures “snapshots” of what you’re doing on your Windows PC at regular intervals, then stores, scans and indexes those screenshots for semantic search.
Recall will require at least 25GB of free space to function (though you can allocate more if you want) and will be turned on by default on PCs that support it, but you can pause it temporarily or turn it off entirely through system settings. I can see why you’d want to turn it off, too, because Recall will use its own logic to decide when to capture snapshots of your screen and save them.
When I spoke with Microsoft Surface exec Pete Kyriacou at Build he told me Recall is designed to only take snapshots when you do something new or different on your PC, which could be clicking a link on a webpage, navigating to a different window or launching a new app. These snapshots of your screen are scanned and indexed by Windows, so when you later try to search for a word, phrase or basic descriptor (“the vacuum I was shopping for on Wednesday,” for example) Recall will use AI inferencing to scan those screenshots (which Microsoft calls “screenrays,” a la an X-ray of your screen) and show the relevant ones to you in a scrollable timeline.
When you search for something in Recall, the results appear in a window with a dated timeline at the top, and you can scroll back and forth across to see the screenrays Recall has captured. When you find what you’re looking for you can click on it to be taken immediately to that website, file, or whatever else you’re looking for.
While this currently only works with Windows apps, Kyriacou told me that the plan is to release a Recall API and work with app developers to help them integrate Recall functionality into their products, with the goal of eventually being able to click on snapshots of those apps and be instantly taken back in time to whatever you were doing in Spotify or Photoshop when Recall recorded a snapshot.
According to Microsoft, the snapshots Recall captures are encrypted by default and stored and scanned locally, so there should be no risk of your sensitive data going out to a remote server for processing. You can also delete your snapshots at any time.
Still, it’s a little terrifying to know Windows can constantly watch everything you’re doing and take AI-parsable pictures at key intervals.
Cocreator
Cocreator is a new feature of Paint in Windows 11 that’s only available on Copilot+ PCs, and it basically allows you to tell Paint what kind of art you want to make, and then fine-tune how much Paint applies that request to the artwork you create.
To be clear, Cocreator doesn’t generate an image in Paint for you—that kind of image generation must still be done by Copilot or other generative AI programs. Instead, when you click the Cocreator button in Paint a control panel appears off to the side with a text input window and a few sliders.
The text input window asks you to “describe what you’d like to create,” and when you type in a simple description (something as simple as “cyberpunk city” or as specific as “a cute turtle, swimming under the ocean, with coral reefs, fish and dramatic lighting”) it uses AI to apply your request to whatever you actually draw in Paint.
I saw this live in person at Build, and it basically works like a filter: You draw your art in Paint, type in what you want it to look like in Cocreator, and then the app automagically tries to turn whatever you drew into whatever you were trying to draw.
The results are mixed, in my brief experience seeing hands-on demos at Build. You can fine-tune the results using a set of controls in Cocreator, which right now amount to a slider you can use to control how hard Cocreator goes in modifying your art as well as a “Choose a style” dropdown menu with options like pixel art or oil painting.
This feature was teased in 2023 and rolled out to Windows Insiders for testing shortly thereafter, so it’s not really new. It’s also unclear whether it will only be available on Copilot+ PCs in perpetuity, or if PCs which don’t meet the Copilot+ PC requirements will still be able to access it.
Live Captions
Live Captions is a great example of the kind of confusion you may encounter when deciding whether to buy a Copilot+ PC, because it’s a feature that’s already available in Windows 11. But with a Copilot+ PC, it seems as though Live Captions will work a bit better than on an older Windows 11 PC that doesn’t meet the Copilot+ PC requirements.
While Live Captions already works in Windows to provide captions in English for any audio playing on your PC (from a Google Meet call to a YouTube video or Twitch stream), the version of Live Captions I saw running on Copilot+ PCs at Build is more capable and can do real-time translation in addition to captions.
This means that while you can currently enable Live Captions in Windows 11, it can only caption in English and it can’t translate other languages into English in real time. But with a Copilot+ PC, the improved Live Captions feature can translate any live or pre-recorded audio into English from over 40 languages, display the subtitles in English instantly and even when you’re offline.
Plus, there’s the persistent promise that a Copilot+ PC will be able to run Live Captions better than a Windows PC without an NPU because Windows 11 can offload the live translation/subtitling work to the NPU, leaving your CPU free to focus on other tasks.
Windows Studio Effects
The Windows Studio Effects built into Windows 11 will also get some upgrades on Copilot+ PCs, and while they aren’t terribly exciting there was one that caught my attention at Build 2024: A new teleprompter feature that should make it easier to read from a script during meetings.
If that’s not the most exciting new feature you’ve ever heard of, I get it. But I saw it live in action at Build, and I was impressed at how such a small feature can take so much pressure and stress out of your life. I personally struggle with feeling cool, calm and collected while leading meetings or delivering pre-written remarks, and this new teleprompter feature could be a big help in those situations.
It’s effectively an expanded version of the eye contact feature already available in Windows Studio Effects, which subtly manipulates your video feed to make it look as though your eyes are always focused on the camera, even when you look away.
I’ve never liked this feature since it feels creepy and inauthentic, but the addition of a teleprompter on your end suddenly makes me appreciate it a lot more. With the new teleprompter feature you can display text on-screen for you to read, and while you’re doing that the PC subtly manipulates your video to make it appear as though you’re looking at your webcam instead of reading from a script.
It’s a minor upgrade that could be a big deal to some folks, and while I’m not sure why it’s only available on Copilot+ PCs I am sure I’d love to have it the next time I have to give a speech.
Outlook
I don’t know about you, but I’m equal parts impressed and scared by Microsoft’s ongoing push to integrate AI into every aspect of Windows.
When I saw Recall and Cocreator in action it was easy to imagine them rolling out to Windows PCs around the world, and with Recall at least I can see clear value. I’ve developed a handful of lazy techniques for tracking down files and data on my Windows PC, and while they work just fine I can easily see myself using Recall on a weekly basis to look up webpages I visited or find documents with specific text.
But honestly, once I have a chance to test it out for a while I think I’ll disable Recall entirely because I just don’t trust Microsoft or AI in general enough to let them scan and index my activity.
And while Cocreator is neat, I’ve already lost my taste for AI-generated art thanks to the overwhelming explosion of it online. Now that every small business in my neighborhood is using the same AI art generators to create email ads, banners and promo art, the appeal of art created by humans just keeps going up. Like Recall, Cocreator seems neat and potentially useful to some, but it does nothing for me personally.
No, as I headed for Sea-Tac airport to head home from Build I was surprised to find the announcements that meant the most to me were live captions and the new teleprompter feature.
They seem like minor features, but in practice the live captions make the process of using a Windows PC a lot more robust and accessible. And the Teleprompter feature is the first Windows Studio Effect I’ve actually seen a use for, whereas the existing features (like filters, automatic eye contact and background blur) have always felt like cosmetic frivolities which are nice to have.
But there are almost certainly more Windows AI features coming from Microsoft in the year ahead, some of which will likely be exclusively available on Copilot+ PCs. So if these additions didn’t do anything for you, stay tuned—the AI PC craze shows no signs of slowing in 2024 or 2025.
:max_bytes(150000):strip_icc()/roundup-writereditor-loved-deals-tout-f5de51f85de145b2b1eb99cdb7b6cb84.jpg "I’m a Travel Writer, and Out of the 5 Million Prime Day Deals on Site, These Are the 12 I’m Shopping")
