Notifications
Clear all

understanding which plugin in a fat binary gets loaded

3 Posts
2 Users
0 Likes
58 Views
(@dolmensdude)
Eminent Member
Joined: 4 months ago
Posts: 16
Topic starter  

also continuing here at Shane's request instead of FB forum.


   
Quote
(@getdunne)
Illustrious Member Admin
Joined: 4 years ago
Posts: 4292
 

What the heck is a "fat binary"?

In 1994, when Apple first transitioned the Macintosh from the Motorola 68000 processor architecture to the then-new IBM PowerPC architecture, they faced a problem. Their entire MacOS operating system was written for the 68000, and they didn't have enough time to rewrite it all, so they devised a way to allow the PowerPC chip to emulate the 68000 chip (a well-known trick), and furthermore made this work with arbitrary combinations of "native" PowerPC code and "emulated" 68000 code (probably the single most epic hack in the history of computing). This allowed them to rewrite only the most time-critical parts of the MacOS operating system, and allow the rest to run in emulation. Over several ensuing years, they gradually rewrote more and more of the OS, until eventually it was all "native".

This clever emulation hack also meant that every third-party program that had been compiled for older Macs automatically ran on the new ones, but developers were keen to rebuild their apps to run natively on the newer PowerPC processors, and Apple wanted to encourage this, but there was another problem. Apple was still selling a mix of PowerPC and 68000 Macs, and of course most Mac users still had the older ones. Neither Apple nor the developers wanted to have to publish separate 68000 and PowerPC versions of their apps, because many users wouldn't understand the difference and might install the wrong one by mistake. The solution was to allow apps to be published in a new "fat binary" format, containing two separate compiled versions of the code, one for 68000, one for PowerPC. When MacOS went to run an application, it looked in the app binary for a version compiled for the Mac's processor architecture. If it found one, it would run it natively; otherwise, it ran the one it found in emulation. Presto, apps could be distributed in a single format, and would automatically run on both older and newer Macs, taking full advantage of the newer processors whenever possible.

In 2006, when Apple transitioned again from PowerPC to Intel processors, they were well prepared. They devised a newer "fat binary" implementation (the first one was a bit of a hack), this time supporting PowerPC and Intel architectures, and created a new emulation software called Rosetta to emulate PowerPC code on Intel processors. In the newer fat-binary format, an app (or plug-in) looks and acts like an ordinary file, but is actually a so-called bundle--a folder disguised as a single file, containing the multiple compiled versions of the code, plus other things like icons.

"Apple Silicon" aka ARM aka M1, M2, etc.

By the time Apple decided to transition the Mac from Intel processors to their own "Apple Silicon" designs, based on the ARM architecture, all this was old hat. Today's "Apple Silicon" Macs use the arm64 (64-bit ARM) CPU architecture, and the latest MacOS includes Rosetta 2 software, which can emulate the x86_64 architecture on ARM processors. So naturally, today's fat-binaries contain arm64 and x86_64 versions of programs/plug-ins.

The Macs in use today are a mix of Intel and ARM types, which is why we're still publishing "fat binaries". Intel Macs only ever run the Intel versions of apps and plug-ins; ARM aka "Apple Silicon" aka M1/M2/etc. run the ARM versions "natively" when available, or the Intel versions via Rosetta 2 otherwise. It's even possible through the Finder's "Get Info" function, to check a box to force a fat binary app to run via Rosetta 2, as though it didn't contain an ARM version of the code. This can be quite handy for testing. MacOS includes an "Activity Monitor" program which lists all running programs in a table which includes a "Kind" column showing "Apple" or "Intel", to indicate which type of code is running. This is sometimes referred to as the "execution mode".

On an Intel Mac, everything runs in "Intel" mode, which is the "native" mode for its Intel processor. On an ARM Mac, these days, most app-bundles contain an arm64 version of the code, so they run natively in "Apple" mode. Older app-bundles may contain only an x86_64 version of the code, in which case they run in "Intel" mode, using Rosetta 2 emulation. (Really old app bundles may contain 32-bit x86_32 versions, which older Intel versions of MacOS would also run, but the latest versions won't.)

Rosetta 2 and plug-ins

When an app tries to load a plug-in, it can ONLY run it in the current execution mode--Rosetta 2 doesn't support the mixed execution-mode trick that Apple pulled off on the first PowerPC Macs. Plug-ins can also be published as fat binaries, and so in many cases this is fine, but many are not. For example, plug-ins built before 2020 (the vast majority of classic ones) are Intel-only.

Apple realized this was going to be a huge problem, so they built a new kind of mixed-mode trick into MacOS for the new Apple Silicon Macs. When an app like Logic Pro X, running natively in Apple mode, goes to load an Audio-Unit (AU) plug-in, MacOS checks the plug-in bundle contents. If an arm64 version is available, it runs it as usual. If not, it fires up a separate program called AUHostingService, running in Intel mode, loads the Intel-only plug-in into that, and Logic loads an ARM-native "proxy" or "bridge" AU plug-in which communicates with it.

(The very latest version of Logic Pro actually uses bridging for ALL plug-ins, so if you run Activity Monitor you'll see AUHostingService instances running in Apple mode. This provides enhanced stability, because if the plug-in causes a crash, only the AUHostingService process crashes, while Logic itself displays a nice friendly error message but keeps running.)

Unfortunately, Apple only chose to provide this built-in bridging magic for Audio-Unit plug-ins, which is no problem for them, because their audio apps (Logic Pro, Mainstage, GarageBand) only use AU plug-ins. However, it created a HUGE problem for me, because Unify supports VST and VST3 plug-ins and most people use them, for patches that must work on both Mac and Windows (where AU is not supported at all). It took me eight months to devise, implement and test our solution, which uses our own AU bridging plug-in (called UniWrap).

So, to answer your original question...

After all of the above preamble, I can finally answer @dolmensdude's original question (posed on Facebook):

When loading a patch that has the VST version of the plugin used for a patch, does Unify load the Intel version of the fat binary? or does it say "hey, let's load the native one?".

UniWrap is an Intel-only AU plug-in. When Unify (running in Apple mode) goes to load, say, a VST or VST3 plug-in, it first tries to do so normally. If the plug-in contains arm64 code, this will succeed, and the plug-in will run in Apple mode. If it fails, Unify loads UniWrap instead. Because UniWrap is an Intel-only, Audio-Unit plug-in, Apple's magic AU bridging kicks in and runs it inside an AUHostingService instance under Rosetta 2. Unify sends a message to the new (emulated) UniWrap instance, specifying the details about the Intel-only VST or VST3 plug-in it originally tried to run, and UniWrap (which IS running in Intel mode) loads that plug-in and translates all the communication back and forth to Unify, so it behaves exactly like that VST/VST3.

When Unify goes to load an AU plug-in, Apple's AU bridging logic applies: If the plug-in bundle contains arm64 code, it loads normally and runs in Apple mode. Otherwise, the x86_64 code gets executed (in Intel mode, i.e., emulated in Rosetta 2) inside an AUHostingService instance.


   
ReplyQuote
(@dolmensdude)
Eminent Member
Joined: 4 months ago
Posts: 16
Topic starter  

Okay, so an ARM VST is in fact loading. The plugin itself, if it is so kind as to display this info, depicts that even if in Unifyks display it shows x64. A manual load of the same VST plugin does not display x64 behind the plugin name? Then it’s purely cosmetic and all is as it should be.

 

some amazing stuff! Thanks for this!

 


   
ReplyQuote
Share: