Apple Intelligence, Apple’s new generative AI offering, is not just limited to consumers. During Apple’s keynote address at WWDC 2024, the company revealed that developers will also have the opportunity to integrate Apple Intelligence into their own apps. To facilitate this, Apple has updated its software development kits (SDKs) with new APIs and frameworks.
One of the exciting features that developers can incorporate into their apps is the Image Playground, or GenAI image creation. With just a few lines of code, apps like Craft can now enhance user documents by allowing them to add AI-generated images. This feature opens up a world of possibilities for developers to make their apps more visually engaging.
In addition to image creation, Apple has also introduced AI-powered writing tools. These tools will be automatically available in any app that utilizes the standard editable text view. For example, Bear Notes, an app for taking notes, will now allow users to rewrite, proofread, and summarize their notes using AI. This integration of AI into various apps will undoubtedly improve the user experience and productivity.
Furthermore, Apple is expanding Siri’s capabilities for developers. Those who have already adopted SiriKit, an SDK for integrating Siri into their apps, will automatically benefit from Siri’s new enhancements. These enhancements cover areas such as Lists, Notes, Media, Messaging, Payments, Restaurant reservations, VoIP calling, and Workouts. Developers can leverage these enhancements without any additional work on their part.
Two new Siri capabilities are particularly noteworthy. Firstly, Siri can now invoke any item from an app’s menus. This means that users can simply ask Siri to perform specific actions within an app, such as showing presenter notes or accessing speaker notes. Secondly, Siri can access any text displayed on the screen using Apple’s standard text systems. This allows users to reference and act on text seamlessly. For example, if you have a reminder or note to wish your grandpa a happy birthday, you can simply say “FaceTime him” to take action on that note.
Apple’s App Intents framework, which enables lightweight app-like interactions without the need for installing the app, is also incorporating Apple Intelligence. Developers across various categories, such as Books, Browsers, Cameras, Document Readers, File Management, Journals, Mail, Photos, Presentations, Spreadsheets, Whiteboards, and Word Processors, will have access to these intents. Apple is working to define and test these intents to ensure they are easily adoptable by developers. For example, a photo-editing app like Darkroom can leverage the Apply Filter intent, allowing users to simply say “Apply a cinematic preset to the photo I took of Ian yesterday” to take action within the app. Apple plans to expand the number of domains supported by App Intents over time.
Apple also announced that its password-manager app, AI-generated Bitmoji, and Calculator for the iPad are now available.
Overall, Apple’s introduction of Apple Intelligence and the integration of AI capabilities into apps will revolutionize the way users interact with technology. Developers now have the tools and resources to enhance their apps with AI-powered features, making them more intuitive and user-friendly. With Siri’s expanded capabilities and the inclusion of Apple Intelligence in various app domains, users will experience a more conversational and seamless interaction with their devices. The future of app development is undoubtedly AI-driven, and Apple is at the forefront of this innovation.