Apple Intelligence is what Apple is calling its artificial intelligence feature set, previewed at WWDC in June 2024. The first Apple Intelligence features launched with iOS 18.1, iPadOS 18.1, and macOS Sequoia 15.1, but this is only an initial taste of what Apple has planned.
There are a selection of Apple Intelligence features that are available now, along with additional capabilities that will be rolling out across the rest of 2024 and 2025.
Apple Intelligence Features Available Now
Writing Tools
- Proofread text checks for spelling and grammar errors, including word choice and sentence structure. You can accept all suggestions with a tap or go through them one by one with explanations.
- Rewrite cleans up what you’ve written and shifts the tone without impacting your content. Options include Friendly, Professional, and Concise.
- You can select text and get a summary of it with Apple Intelligence. You can choose to create a paragraph, pull out key points, make a list, or create a table. Summaries are available in Mail, Messages, and more.
You can select any text on your iPhone, iPad, or Mac and use Apple Intelligence to access Writing Tools for summaries and other features.
Siri
- There’s a new glow around the edges of the display when Siri is activated, applicable to iPhone, iPad, and CarPlay. On Mac, the Siri window can be placed anywhere. The glow animates responsively to the sound of your voice so you can tell when Siri is listening without interrupting other things you’re doing.
- A double tap at the bottom of the display brings up the Type to Siri interface so you can type requests instead of speaking them. On Mac, you need to press the Command key twice to bring up Type to Siri. Type to Siri includes suggested requests so you can get your questions answered faster.
- Siri can maintain context between requests so you can ask a question and then reference it in a second reply. If you ask about the temperature in Raleigh, for example, and then follow up with “what’s the humidity?” Siri should know you mean in Raleigh.
- If you stumble over your words when speaking to Siri, or change what you’re saying mid-sentence, Siri will follow along.
- Siri has Apple’s product knowledge and support base for answering questions about your device’s features and settings, and can even find settings when you don’t know the exact name by using natural language search.
- There is a summarize button for summarizing any of your incoming emails, plus you will see a brief summary of an email in your inbox list rather than the first few lines of the email.
- Mail surfaces time sensitive messages first when applicable, putting them at the top of your inbox so you see what’s important right away.
- Smart Reply provides quick-tap responses to emails that you’ve been sent, with contextual options based on what’s in the email.
- Multiple notifications from Mail will be summarized on your Lock Screen so you can see what’s in an email without opening the app.
Messages
- Messages has Smart Reply options for incoming texts, which analyze the content of messages to offer suggestions of what you might want to say.
- Multiple Messages notifications are summarized on your Lock Screen.
- You can use all of the Writing Tools features in the Messages app for proofreading and refining what you’re planning to send.
Photos
- You can create a Memory Movie with just a description, such as “My cat in 2024,” or “Orlando in the summer.” The feature automatically picks relevant photos and chooses songs, but you can tweak through the Memory Mixes feature or choose a mood to guide the direction of the audio. You can also add in specific scenes and images you want to see throughout the memory when you’re creating the prompt.
- Natural language search is available in Photos, so you can just describe what you’re looking for, such as “Eric rollerskating while wearing green.”
- Search can also find specific moments in video clips.
- Search offers up smart complete suggestions for narrowing down what you might want to find.
Clean Up
The Photos app also includes “Clean Up,” a feature that lets you remove unwanted objects from your photos. The Clean Up tool in the Photos app is able to automatically detect objects in an image that might not be wanted, but you can also tap, circle, or brush over an unwanted object to remove it.
Zooming in on an image can help with using a finger as a brush to remove smaller blemishes and issues with an image, and it is intelligent enough not to remove part of a person even if a person or main subject is selected.
Clean Up works on all images in the Photos library, including older images and images captured by other devices like a point and shoot camera or a DSLR.
Transcription Summaries
In Notes and other apps, you can record audio and get a transcript along with a summary of your transcript, which is useful for recording lectures and other audio. Transcription isn’t an Apple Intelligence feature, but summaries are.
Focus Modes
There is a dedicated Reduce Interruptions Focus Mode that only shows you important notifications that need attention while filtering out everything else.
When customizing an existing Focus mode or creating a new one, there is a new toggle to turn on Intelligent Breakthrough and Silencing. This option allows important notifications to interrupt you, while muting notifications that are not important, which is the same thing that the Reduce Notifications Focus does. You can override the notification settings for specific apps and people, which is how it worked prior to iOS 18.1.
Notification Summaries
Your incoming notifications are summarized so you can see what’s new for each app at a glance.
Phone
Summaries of transcriptions generated from your phone calls are supported in iOS 18.1. The iOS 18.1 update adds the option to record a phone call and get a transcription, which is not an Apple Intelligence feature. What does require Apple Intelligence, though, is getting a summary from that transcription without reading through the entire thing.
Note that you can start a recording by tapping on the record button in the upper left corner of the display when on a phone call. All participants are notified that the call is being recorded before the recording starts.
Recorded phone calls are stored in the Notes app, where you can tap in to view a transcript and get a summary generated from that transcript.
Safari
When reading an article in Reader Mode, there is an option to have Apple Intelligence summarize the article for you.
Apple Intelligence Features Coming in iOS 18.2
The next set of Apple Intelligence features will come in iOS 18.2, iPadOS 18.2, and macOS Sequoia 15.2, and this is when Apple plans to release Image Playground, Genmoji, and ChatGPT Siri integration. These features are being beta tested right now.
Apple has a waitlist for Image Playground, Genmoji, and Image Wand, and these features will be rolling out to developers over the coming weeks so that Apple can collect feedback and make improvements to the image generation capabilities before a public launch later this year.
Image Playground
Image Playground can be used to generate images in cartoon-like styles using a text-based prompt. Apple has built-in suggestions and concepts that you can choose from, like costumes, locations, items, and themes.
While Image Playground is a standalone app, it is also integrated into the Messages and Notes app. In these apps, Apple can use context from what you’ve typed for image generation suggestions.
For inspiration, you can upload or take a photo, and then get a cartoon AI version of it, and you can also use images of friends and family members pulled from the People album in the Photos app.
You can start with a base suggestion or photo and then continue to add to it until you get what you want. You can remove suggestions at any time, and save your favorite creations for use in other apps. Anytime you create something with Image Playground, you’ll get multiple options so you can choose the best one.
There are only Animation and Illustration styles for Image Playground, so there is no option for creating photorealistic images.
Image Wand
Image Wand is basically the same thing as Image Playground, but in the Notes app. When you have notes that you’ve taken, you can circle an empty spot or some text and Image Wand will add a contextually relevant image.
So if you have notes on photosynthesis, you can add in an image of a plant under the sun. Image Wand isn’t able to generate complex images, so if you want a picture of the internal structure of mitochondria, you’re out of luck, but it can make a stylized image featuring the organelle.
On an iPad, you can draw a rough sketch of what you want to add to your notes with an Apple Pencil, and then use Image Wand to generate something more polished.
Genmoji
Genmoji are custom emoji characters that you can create using a text-based description. If there’s an emoji you can’t find but need, like a duck eating a sandwich or an alligator skateboarding, Genmoji can make it for you.
Genmoji aren’t too far off from Image Playground images, but the generation system tends to want to add a person for a lot of requests. You can choose yourself or a friend or family member, or just use a generic emoji character as your base.
In Messages and other apps, Genmoji behave like emoji, but they’re not going to display properly for anyone running an operating system earlier than iOS 18.1, or on an Android device. In iOS 18.1 and iOS 18.2, they work like emoji, but Genmoji are displayed as a blank box and an accompanying full-size image on other versions of iOS.
If you have iOS 18.1 and iOS 18.2 and someone sends a Genmoji, you can long press on it and tap the “Emoji Details” button to see what prompt was used to create it, plus you can add it to your own Genmoji/sticker collection to use.
To create a Genmoji character, tap into the Emoji keyboard and tap on the Emoji with a “+” button next to the search bar. From there, you can type in your idea.
Siri ChatGPT Integration
Siri ChatGPT integration lets Siri hand requests over to OpenAI’s ChatGPT. ChatGPT is off by default, but you can turn it on in the Apple Intelligence and Siri section of the Settings app.
If ChatGPT integration is enabled, Siri will consult ChatGPT for complex requests. Complex requests might include creating an image, generating text from scratch, making recipe ideas based on what’s in your refrigerator, describing what’s in a photo, and more.
Siri will analyze each request to see if it’s something that needs to be answered by ChatGPT, but you can also automatically invoke ChatGPT for a request by using a request like “Ask ChatGPT to give me a chocolate chip cookie recipe.”
Siri will ask your permission before querying ChatGPT, but there is an option to turn off that extra permission step by toggling off “Confirm ChatGPT Requests” in the ChatGPT section of Settings.
You don’t need an account to use ChatGPT, and it is free, but if you have a paid account, you can sign in. If you’re not signed in, OpenAI does not store any of your ChatGPT requests, nor is your information used for training ChatGPT. If you sign in, ChatGPT can save a copy of your queries. Apple does not store ChatGPT queries.
ChatGPT can be used with Siri, but it is also integrated into Writing Tools and Visual Intelligence. With Writing Tools, ChatGPT can generate text, and with Visual Intelligence, ChatGPT can answer questions about what the Camera sees.
Visual Intelligence
Visual Intelligence is an iPhone 16 feature that uses the Camera Control button. If you long press it, you can get into Visual Intelligence mode, where the Camera app can be used to identify what’s around you.
If you point the camera at a store, for example, you can see ratings, hours, and other information. If you take a photo of an object, you can get more information about the object from ChatGPT, or use it with Google Search to find similar images. The Google Search feature is a good way to search for products that you want to find.
Other Visual Intelligence features include reading text out loud, detecting phone numbers and addresses to add them to the Contacts app, copying text, and summarizing long passages of text.
Writing Tools
Writing Tools is in iOS 18.1, but in iOS 18.2, you can make more open-ended changes to what you’ve written. You can come up with your own tone changes, so if you want something to be more flowery or elaborate, Writing Tools can make it happen. You can also ask for your text to be converted into a different format, like a poem.
Writing Tools also has ChatGPT integration in iOS 18.2 so if…
