Explore the integration of media technologies within your app. Discuss working with audio, video, camera, and other media functionalities.

Posts under General subtopic

Post

Replies

Boosts

Views

Created

Terrible performance when using MusicKit's Artwork with UIKit
I'm building a UIKit app that reads user's Apple Music library and displays it. In MusicKit there is the Artwork structure which I need to use to display artwork images in the app. Since I'm not using SwiftUI I cannot use the ArtworkImage view that is recommended way of displaying those images but the Artwork structure has a method that returns url for the image which can be used to read the image. The way I have it setup is really simple: extension MusicKit.Song { func imageURL(for cgSize: CGSize) -> URL? { return artwork?.url( width: Int(cgSize.width), height: Int(cgSize.height) ) } func localImage(for cgSize: CGSize) -> UIImage? { guard let url = imageURL(for: cgSize), url.scheme == "musicKit", let data = try? Data(contentsOf: url) else { return nil } return .init(data: data) } } Now, everytime I access .artwork property (so a lot of times) the main thread gets blocked and the console output gets bombared with messages like these: 2023-07-26 11:49:47.317195+0200 Plum[998:297199] [Artwork] Failed to create color analysis for artwork: <MPMediaLibraryArtwork: 0x289591590> with error; Error Domain=NSCocoaErrorDomain Code=4099 "The connection to service named com.apple.mediaartworkd.xpc was invalidated: failed at lookup with error 159 - Sandbox restriction." UserInfo={NSDebugDescription=The connection to service named com.apple.mediaartworkd.xpc was invalidated: failed at lookup with error 159 - Sandbox restriction.} 2023-07-26 11:49:47.317262+0200 Plum[998:297199] [Artwork] Failed to create color analysis for artwork: file:///var/mobile/Media/iTunes_Control/iTunes/Artwork/Originals/4b/48d7b8d349d2de858413ae4561b6ba1b294dc7 2023-07-26 11:49:47.323099+0200 Plum[998:297013] [Plum] IIOImageWriteSession:121: cannot create: '/var/mobile/Media/iTunes_Control/iTunes/Artwork/Caches/320x320/4b/48d7b8d349d2de858413ae4561b6ba1b294dc7.sb-f9c7943d-6ciLNp'error = 1 (Operation not permitted) My guess is that the most performance-heavy task here is performing the color analysis for each artwork but IMO the property backgroundColor should not be a stored property if that's the case. I am not planning to use it anywhere and if so it should be a computed async property so it doesn't block the caller. I know I can move the call to a background thread and that fixes the issue of blocking main thread but still the loading times for each artwork are terribly slow and that impacts the UX. SwiftUI's ArtworkImage loads the artworks much quicker and without the errors so there must be a better way to do it.
5
0
1.5k
Jul ’23
ScreenCaptureKit and mixed Retina/non-Retina configuration
The two ScreenCaptureKit WWDC22 sessions show how to capture with the new framework but the retina factor is hardcoded to 2 in SCStreamConfiguration. When using on a non-retina display, the screencapture is floating on the upper-left corner of the image buffer. There does not seem to be a simple way to retrieve the retina factor from the SCShareableContent data (when configuring the capture). When processing the streaming output, the SCStreamFrameInfo attachment is supposed to have a scaleFactor property but .scaleFactor does not return a value. I have found out that the attachement dictionary contains SCStreamUpdateFrameDisplayResolution. This entry gives me the retina factor but it is not an official SCStreamFrameInfo key. I list the keys to access it. What is the proper way with ScreenCapture to handle the retina factors ?
1
1
823
Feb ’23
Ventura Hack for FireWire Core Audio Support on Supported MacBook Pro and others...
Hi all,  Apple dropping on-going development for FireWire devices that were supported with the Core Audio driver standard is a catastrophe for a lot of struggling musicians who need to both keep up to date on security updates that come with new OS releases, and continue to utilise their hard earned investments in very expensive and still pristine audio devices that have been reduced to e-waste by Apple's seemingly tone-deaf ignorance in the cries for on-going support.  I have one of said audio devices, and I'd like to keep using it while keeping my 2019 Intel Mac Book Pro up to date with the latest security updates and OS features.  Probably not the first time you gurus have had someone make the logical leap leading to a request for something like this, but I was wondering if it might be somehow possible of shoe-horning the code used in previous versions of Mac OS that allowed the Mac to speak with the audio features of such devices to run inside the Ventura version of the OS.  Would it possible? Would it involve a lot of work? I don't think I'd be the only person willing to pay for a third party application or utility that restored this functionality. There has to be 100's of thousands of people who would be happy to spare some cash to stop their multi-thousand dollar investment in gear to be so thoughtlessly resigned to the scrap heap.  Any comments or layman-friendly explanations as to why this couldn’t happen would be gratefully received!  Thanks,  em
63
10
34k
Dec ’22
iOS 16 MusicKit - Artwork has no background or text colors
Hello, I'm new to the Swift MusicKit API and am starting with the implementation in iOS 16. I'm getting stuck on an issue where there is no background or text color associated with the Artwork object. Is this something you have to make an additional property request for, and if so, how do you do that? var catalogSearch = MusicCatalogResourceRequest<Album>(matching: \.id, equalTo: item.id) let catalogResponse = try await request.response() guard let firstItem = catalogResponse.items.first else { return } In this example, firstItem.artwork only contains the url and what look like incorrect max width/height values. here's a printout of firstItem.artwork Optional(Artwork(   urlFormat: "musicKit://artwork/library/5F37858D-F46B-4F12-BA67-40FA8DD63D87/{w}x{h}?at=item&fat=&id=7718670444435992305&lid=5F37858D-F46B-4F12-BA67-40FA8DD63D87&mt=music&aat=Music122/v4/37/25/f5/3725f515-249f-7b91-77bb-f479cd48201c/22UMGIM32254.rgb.jpg", maximumWidth: 0, maximumHeight: 0 ))
1
2
911
Jun ’22
Editing a Library Playlist (MusicKit: iOS 16 beta)
I've just begun to dip my toes into the iOS16 waters. One of the first things that I've attempted is to edit a library playlist using: try await MusicLibrary.shared.edit(targetPlaylist, items: tracksToAdd) Where targetPlaylist is of type MusicItemCollection<MusicKit.Playlist>.Element and tracksToAdd is of type [Track] The targetPlaylist was created, using new iOS16 way, here: let newPlaylist = try await MusicLibrary.shared.createPlaylist(name: name, description: description) tracksToAdd is derived by performing a MusicLibraryRequest on a specific playlist ID, and then doing something like this: if let tracksToAdd = try await playlist.with(.tracks).tracks {    // add tracks to target playlist } My problem is that when I perform attempt the edit, I am faced with a rather sad looking crash. libdispatch.dylib`dispatch_group_leave.cold.1:     0x10b43d62c <+0>:  mov    x8, #0x0     0x10b43d630 <+4>:  stp    x20, x21, [sp, #-0x10]!     0x10b43d634 <+8>:  adrp   x20, 6     0x10b43d638 <+12>: add    x20, x20, #0xfbf          ; "BUG IN CLIENT OF LIBDISPATCH: Unbalanced call to dispatch_group_leave()"     0x10b43d63c <+16>: adrp   x21, 40     0x10b43d640 <+20>: add    x21, x21, #0x260          ; gCRAnnotations     0x10b43d644 <+24>: str    x20, [x21, #0x8]     0x10b43d648 <+28>: str    x8, [x21, #0x38]     0x10b43d64c <+32>: ldp    x20, x21, [sp], #0x10 ->  0x10b43d650 <+36>: brk    #0x1 I assume that I must be doing something wrong, but I frankly have no idea how to troubleshoot this. Any help would be most appreciated. Thanks. @david-apple?
11
0
3.3k
Jun ’22
MPMediaPlayback.currentPlaybackRate no longer working in iOS 15.4?
Just wondering if anyone else is having issues with currentPlaybackRate in release version of iOS 15.4? In my particular case this is using MPMusicPlayerController.applicationQueuePlayer. I've always had issues controlling this property reliably but from what I can see it is now completely non-operational in 15.4. I've isolated this behavior in a trivial project, and will file a radar, but hoping others may have some insight first. FWIW- This is my trivial test case: class ViewController: UIViewController {     lazy var player: MPMusicPlayerApplicationController = {         let player = MPMusicPlayerController.applicationQueuePlayer         player.repeatMode = .none         player.shuffleMode = .off         player.beginGeneratingPlaybackNotifications()         return player     }()     override func viewDidLoad() {         super.viewDidLoad()         NotificationCenter.default.addObserver(forName: .MPMusicPlayerControllerPlaybackStateDidChange, object: nil, queue: .main) { [weak self] notification in             guard let notificationPlayer = notification.object as? MPMusicPlayerApplicationController,                   notificationPlayer === self?.player else {                 return             }                          debugPrint("Player state now: \(notificationPlayer.playbackState)")         }     }     @IBAction func goAction(_ sender: Any) {         guard let item = MPMediaQuery.songs().items?.randomElement() else {             debugPrint("Unable to access media items")             return         }         debugPrint("Now playing item: \(item.title ?? "")")         player.setQueue(with: [item.playbackStoreID])         player.prepareToPlay() { error in             guard error == nil else {                 debugPrint("Player error: \(error!.localizedDescription)")                 return             }             DispatchQueue.main.async { [weak self] in                 self?.player.play()             }         }     } @IBAction func slowAction(_ sender: Any) {         debugPrint("Setting currentPlaybackRate to 0.5")         player.currentPlaybackRate = 0.5         checkPlaybackRate()     } @IBAction func fastAction(_ sender: Any) {         debugPrint("Setting currentPlaybackRate to 1.5")         player.currentPlaybackRate = 1.5         checkPlaybackRate()     } func checkPlaybackRate(afterSeconds delay: TimeInterval = 1.0) {         DispatchQueue.main.asyncAfter(deadline: .now() + delay) {             debugPrint("After \(delay) seconds currentPlaybackRate now: \(self.player.currentPlaybackRate)")         }     } } Typical console output: "Now playing item: I Know You Know" "Player state now: MPMusicPlaybackState(rawValue: 2)" "Player state now: MPMusicPlaybackState(rawValue: 1)" "Setting currentPlaybackRate to 1.5" "After 1.0 seconds currentPlaybackRate now: 1.0" "Setting currentPlaybackRate to 0.5" "After 1.0 seconds currentPlaybackRate now: 1.0"
25
0
9.3k
Mar ’22
MusicKit in playgrounds not supported?
Has anyone been able to successfully use MusicCatalogSearchRequest in a playgrounds app? I have configured my playground similar to a regular app: app id with automatic music token generation turned on, music access authorized within the app itself, but whenever I query MusicCatalogSearchRequest I get an error thrown with .developerTokenRequestFailed. Considering musickit is restricted in the sim, it would not surprise me if it was the same in playgrounds but it would be super helpful if I could prototype with musickit in playgrounds 4!
2
1
1.3k
Dec ’21
MusicKit WebAPI - Can only get storefronts. All other endpoints return 500.
If I make a request to https://api.music.apple.com/v1/storefronts/us with the proper developer jwt token in the header, I receive the a successful response with a list of store fronts. If I remove the token, I do get back a 401 error. If I call any other catalog base query, I am getting back a 500 error. For instance: https://api.music.apple.com/v1/catalog/us/albums/310730204 returns a 500 error with the body being {"message":"An unexpected error occurred"} I'm not sure what I can do to fix this. Please help.
15
1
4.3k
Oct ’21
How to detect a song end?
I'm playing library items (MPMediaItem) and apple music tracks (Track) in MPMusicPlayerApplicationController.applicationQueuePlayer, but I can't use the actual Queue functionality because I can't figure out how to get both media types into the same queue. If there's a way to get both types in a single queue, that would solve my problem, but I've given up on that one. Because I can't use a queue, I have to be able to detect when a song ends so that I can put the next song in the queue and play it. The only way I can figure out to detect when a song ends is by watching the playBackState, and I've actually got that pretty much working, but it's really ugly, because you get playBackState of paused when a song ends, and when a bluetooth speaker disconnects, etc. The only answer I've been able to find on the internet is to watch the MPMusicPlayerControllerNowPlayingItemDidChange, and when that fires, and the nowPlayingItem is NIL, a song ends.. but that's not the case. When a song ends, the nowPlayingItem remains the same. There's got to be an answer to this problem, right?
14
3
5.3k
Aug ’21
P3 Display to XYZ Color Space Conversion
When I use the ColorSync Utility to convert Display P3 color (1, 0, 0) to an XYZ color, the result is (0.5151, 0.2412, -0.0011). I expected that result because that is identical to the red colorant tristimulus value in the Display P3.icc file.When I use the CGColor converted method to do the same, the XYZ color is approximately (0.5151, 0.2412, 0.0). Note that the third element is 0.0 whereas it is -0.0011 when using the ColorSync Utility. I have printed out the Z component to 16 digits of precision, and Z is all 0s. It appears that the CGColor converted method is clamping the result from 0 to 1.My questions are:1. Which conversion is correct? The ColorSync utility or the CGColor converted method?2. I am not a color specialist, but I thought that the XYZ components should never be negative. If so, is the colorant tristimulus value in the Display P3.icc file wrong?3. Because CGColor clamped the Z component to 0, the XYZ color cannot be converted back exactly or closely to the Display P3 color (1, 0, 0). I would have expected to be able to go back and forth between the two color spaces when starting from a valid P3 Display color especially since the XYZ color space completely encompasses the P3 Display color space. Is that not true?4. Is (1, 0, 0) an invalid Display P3 color? If so, I can understand the peculiar results. I'm not sure how I would know if a Display P3 color is valid or not. (I only know that the component values must be from 0 to 1.) I think it is valid because Apple uses that color in the UIColor API Reference in an example.
3
0
3.4k
Oct ’16