Notifications
Clear all

Suggestion for making "Unified" libraries for plugins

36 Posts
8 Users
13 Likes
2,087 Views
JeremyH
(@zinct)
Member
Joined: 3 years ago
Posts: 514
Topic starter  

I see from the latest PG email that there are some more plugins that have been "Unified" both by end users and by PG. This is great news and something to be encouraged.

There are a growing number of these and I wondered whether there should be a single thread here to list those that are completed (with links) and those being worked on by PG and end users. This is mainly to avoid duplication of effort. I appreciate that there may be some PG ones that cannot be mentioned for commercial reasons.

For example, I thought I might have a go at Unifying Parawave Rapid as I have just bought it but want to make sure that no-one else is doing it. 

A separate hints and tips post might also be useful to document things like patch naming, streamlining the process and what/where to submit their Unified library for review (I assume by PG to ensure quality control). 


   
SSquared reacted
Quote
(@getdunne)
Illustrious Member Admin
Joined: 3 years ago
Posts: 3907
 

@zinct

Hear, hear! I've suggested to John that we start a new "Unified Libraries" forum for exactly this.


   
ReplyQuote
JeremyH
(@zinct)
Member
Joined: 3 years ago
Posts: 514
Topic starter  

@getdunne that's good news, thanks.

After watching last night's Livestream this morning I thought I would make a start with a PG library.

So I own the Kontakt versions of MegaMagic Bells/Winds and MegaMagic Guitars and thought I would try the former.

Now I am just wondering whether it's possible to force Unify to load Kontakt for the first Kontakt patch and subsequently only load patches into the existing instance of Kontakt and sort out the MIDI channel mapping? In other words load patches into the existing Kontakt instance to avoid loading multiple copies of Kontakt and wasting memory. That's probably not easy so I will just assume that it loads new instances of Kontakt for every patch.

I have named the library PluginGuru K5 MegaMagic Bells/Winds and for example the first patch is BELL - Baby Bell. Does that sound right?

This post was modified 2 years ago by JeremyH

   
ReplyQuote
(@getdunne)
Illustrious Member Admin
Joined: 3 years ago
Posts: 3907
 

@zinct

The forthcoming Unify v1.2.x greatly reduces the re-load time for third-party plug-ins. I originally looked into the approach you suggest (re-using existing instances) but it's surprisingly difficult. It turns out that simply loading a new patch before (not after) deleting the current one yields 99% of the speed benefit with 1% of the work.


   
ReplyQuote
(@getdunne)
Illustrious Member Admin
Joined: 3 years ago
Posts: 3907
 

@thsve

The .unify files basically are XML, wrapped with an 8-byte header and a single trailing NUL (zero) byte. The header consists of the four characters 'VC2!' followed by a 4-byte little-endian integer, whose value is the number of bytes in the XML payload. You can edit the XML, but to make the resulting file valid, you'll need to adjust the 4-byte count in the header to match.

What's your preferred scripting language? Perhaps I can help you with some code.


   
ReplyQuote
(@getdunne)
Illustrious Member Admin
Joined: 3 years ago
Posts: 3907
 

@thsve

The attached zip file contains two Python3 scripts, one to unpack a .unify file to a .xml file, and the other to re-pack the edited .xml file.

Note the excellent, free Notepad++ text editor has built-in functions to reformat ("pretty-print") XML to make it more legible, as well as a "linearize" function to jam it all back into a single line when you're done editing it.


   
JeremyH reacted
ReplyQuote
(@getdunne)
Illustrious Member Admin
Joined: 3 years ago
Posts: 3907
 

@thsve

The unpack script already works in Python 2.7.12. A 2.7-compatible version of the repack script is attached.


   
ReplyQuote
(@getdunne)
Illustrious Member Admin
Joined: 3 years ago
Posts: 3907
 

@thsve

The genre tag is something we had in early versions of Unify, but isn't used at the moment. It remains in the patch format, and there's a column for it in the patch database, in case we decide to start tagging patches according to music genres again in future.

If your Omnisphere files have genre tags, you can copy these over if you want, and they'll be incorporated into Unify's patch database. As I said, the genre field isn't used at the moment, but it's harmless to populate it.


   
ReplyQuote
Holmwoody
(@holmwoody)
Eminent Member
Joined: 3 years ago
Posts: 37
 

I am working on Unifying more libraries (I’ve already done several for JL) and I think a public list of those being worked on would be great.

I am doing the MM guitar series currently - should be done soon.

I could then work on Del Norte if no-one else is.


   
ReplyQuote
JeremyH
(@zinct)
Member
Joined: 3 years ago
Posts: 514
Topic starter  
Posted by: @holmwoody

I am working on Unifying more libraries (I’ve already done several for JL) and I think a public list of those being worked on would be great.

I am doing the MM guitar series currently - should be done soon.

I could then work on Del Norte if no-one else is.

I hope to finish the Kontakt version of MM Bells and Winds today. Hoping that you have done the Omnisphere version.

This post was modified 2 years ago by JeremyH

   
ReplyQuote
Holmwoody
(@holmwoody)
Eminent Member
Joined: 3 years ago
Posts: 37
 

@zinct I’m doing the MM guitars, the O2 winds/bells are down the list.


   
ReplyQuote
(@getdunne)
Illustrious Member Admin
Joined: 3 years ago
Posts: 3907
 

There may be a problem with Kontakt libraries. Jeremy (@zinct) sent me his Unified K5 MM Bells/Winds library, and it looks like it might not be possible to do such Unified libraries in a generic way which will work for all users.

I suggest not doing any more Unified versions until we find out more.


   
ReplyQuote
JeremyH
(@zinct)
Member
Joined: 3 years ago
Posts: 514
Topic starter  

Thanks @Holmwoody and @getdunne 

Shane - I am just reading and replying to your email. Your .guru file version works great by the way, thanks!


   
ReplyQuote
(@robert-pryke)
Eminent Member
Joined: 3 years ago
Posts: 43
 

Hello everyone,

 

I have been interested in doing this for automating Omnisphere patches to Unify.

 

Here is how Omnisphere patches are done:

 

We are editing this reply so we do not provide protected information on a public form. Just to be safe. Thank you.


   
ReplyQuote
(@robert-pryke)
Eminent Member
Joined: 3 years ago
Posts: 43
 

@thsve

 

Shane said I should reach out to you regarding your Python script for conversion.

I have added a near nulled out Omnisphere patch to this thread for an example.

 

Hopefully you can take this info and use it to create a converter script in Python


   
ReplyQuote
Holmwoody
(@holmwoody)
Eminent Member
Joined: 3 years ago
Posts: 37
 

@getdunne I believe that John is doing that this week! 


   
ReplyQuote
Holmwoody
(@holmwoody)
Eminent Member
Joined: 3 years ago
Posts: 37
 

When I originally contacted John and Taylor regarding Unifying some pluginguru O2 patch libraries, they gave me the following advice, which seems to negate the idea of simple macros to automate the process:

  • Omnisphere master output should be at 0db
  • Load the Omnisphere Patch and generally people also open the same patch in the standalone version of Omnisphere or somewhere else so they have an easy way to:
  • a) copy over the tags.................... see note below
  • b) copy over the category.............. see note below
  • c) copy over the descriptive text into Unify.

        This is the part that takes time but once you have a system in place, it goes fairly quickly.

  • On the Master effect channel, we're only calling up the LoudMax and setting the threshold at -3db - to be done on each patch.
  • Then save into a folder you create with the naming format of :

         PlugInGuru 02 Name Of the Library V1

  • There is only 1 category usually and that is also in the patch name.
  • The category is spelled regularly not in all caps except "BPM".
  • The main one to get right is the Category. If it is BPM DRUM in the name, when you type the category it will be BPM Drum.
  • Also, try not to include categories in tags section. In this example you would not use BPM Drum as a tag because it is a category.

Don't let me confuse you, you CAN use the tags from omnisphere to SOME extent. (I use only the tags I find in the list in Unify, not the O2 tags - NB)

BUT....... John just doesn't want us to start using the O2 tagging system entirely.............. Keep the tag system Unify specific.

 

I hope that helps a little. John is going to sort out a forum thread this week for listing the libraries we are all working on so that we don't have duplication of effort. There are plenty of libraries to be shared around!!!!

At the moment I am doing all three MM Guitar libraries for O2......... V1, V2 and V12, They will be ready for John to quality control this week and release when he determines it appropriate. There are a lot of other things being released right now or very soon, as we know.

I will then do Del Norte, which should be ready next week for John.

I believe that someone else is currently doing Percussa for O2.

By the way, John would need to get permission for us to Unify non-pluginguru libraries, eg Luftrum, Triple Spiral Audio, etc, so don't even think of Unifying those. It's a serious matter of copyright and those companies have told me they would want to control it (do it) themselves. Plus, once done for one library, they would need to do it for every library in their portfolio - a mammoth task for some. I have no doubt that, given time and the progress and usefulness of Unify, these other library compilers will see the merit of having their libraries Unified, but it will all take time. Be patient. Let's enjoy Unifying and using all the pluginguru libraries first!

Nigel Baillie

 


   
JeremyH reacted
ReplyQuote
(@getdunne)
Illustrious Member Admin
Joined: 3 years ago
Posts: 3907
 

@holmwoody Thank you for the detailed write-up.

Please note the naming for Omnisphere 2 based libraries should be "O2" (letter O) not "02" (zero).

However, I have a program for renaming libraries if the original name is not quite right, so this is not a show-stopper.


   
Holmwoody reacted
ReplyQuote
Holmwoody
(@holmwoody)
Eminent Member
Joined: 3 years ago
Posts: 37
 

@thsve Thanks for explaining your programming paragraph. It had left me bewildered!, lol. It’s good to know we are all part of the same supportive and creative family, headed by John and Shane.


   
JeremyH reacted
ReplyQuote
(@robert-pryke)
Eminent Member
Joined: 3 years ago
Posts: 43
 

@thsve

 

That makes sense.

As you know, .omn files are stored in a text format.  What I was doing with the .omn files was dumping a folder out, using a batch to rename all of the files extensions to .txt.

 

I then was using Excel and Power Query to import the data from the folder and transform the data that way I had a file with the description tag information.  This gives the patch name, the meta data, the author and other information.

I would then go to the Entry Descr section and copy and paste the data into Unify.

Maybe with now knowing that Unify patches are XML we may be able to export the data in directly to convert.

So, if you have your script to extract the data from the patch, maybe Shane can give us a better understanding of how Unify pulls patch data from Omnisphere, that may give us an idea of how to write something that can communicate the data to it.

 

The file structure of a unify file is really the key.

 

 

 

 

 


   
ReplyQuote
(@robert-pryke)
Eminent Member
Joined: 3 years ago
Posts: 43
 

Might be something that Shane is unable to share, as that's what really makes Unify special.

I know he is busy right now.

 

If I get a chance, I'll try to see if I can figure it out, and send any results to Shane first.


   
ReplyQuote
(@getdunne)
Illustrious Member Admin
Joined: 3 years ago
Posts: 3907
 

I'm not going to say anything further on this subject, and indeed I may have to remove this part of the forum entirely, as it could be considered an attempt to reverse-engineer aspects of Spectrasonics software, which is specifically forbidden in their Software License Agreement.

Creating Unified libraries manually, one patch at a time, is not difficult, and enough people are doing it that before long, there will no longer be any point in devising an automated solution.

If you want to continue discussing this further, please do so elsewhere.


   
ReplyQuote
PlugInGuruForums
(@pluginguruforums)
Illustrious Member Admin
Joined: 3 years ago
Posts: 297
 

This has been an interesting discussion - I am going to remove a few of the posts that share sensitive information about plug-ins that we do not own. We don't want to be caught in any sort of controversy with manufactures of software that we respect and support.

I am also going to start a forum for Unifying Libraries. There will be a sticky topic at the top with details about HOW to Unify a Library or Plug-In. Also details about the LARGE CAP Category at the start of our patches - what they stand for. Also details about WHO we can Unify. You CAN NOT Unify a non-PlugInGuru Patch Library or Sample Library without permission from the developer and an agreement that they will include the Unified version of the library in the download they provide for that particular library. You CAN NOT post library patches on a public forum - those are copy written patches and without permission you are breaking copyright law.

Here's the link to the new sticky topic: https://forums.pluginguru.com/postid/4677/

OK - so thank you again for this lively discussion. I'm trying to get Big Bad Beats out the door so a lot of this set up will be matured over time when I have more time.

 

- JL

Life is Sound / Sound is Divine
www.pluginguru.com


   
ReplyQuote
Kendall W Cochran III
(@pbeprod)
Re-Membered
Joined: 3 years ago
Posts: 287
 

@holmwoody hey just checking so you’re doing Del Norte? How far along? I had mentioned to John I was going to but my work schedule is 7 on 7 off and haven’t gotten around to it yet, but if you have started then I’ll probably do Radio Ready, then Mega Magic Dreams. Percussa took a while.

thanks

Kendall W Cochran III


   
ReplyQuote
Holmwoody
(@holmwoody)
Eminent Member
Joined: 3 years ago
Posts: 37
 

@pbeprod hi, like you I have been unexpectedly busy this week. I have nearly finished the MM guitars (x3) collection, but haven’t yet started Del Norte. If you want to do it be my guest, it’s a large library to do. Let me know what you decide, please.


   
ReplyQuote
Kendall W Cochran III
(@pbeprod)
Re-Membered
Joined: 3 years ago
Posts: 287
 

@holmwoody Hey I’d be fine to do it! I’ll start on it Thursday after I wake up. Thankfully it won’t take as long as Percussa it was 847 patches and multis. But Del Norte is only 🙂 515 and 36.


   
ReplyQuote
Holmwoody
(@holmwoody)
Eminent Member
Joined: 3 years ago
Posts: 37
 

@pbeprod Thanks you, that takes a bit of pressure off me. Best wishes. 🙂 


   
ReplyQuote
Kendall W Cochran III
(@pbeprod)
Re-Membered
Joined: 3 years ago
Posts: 287
 

@holmwoody hey just lack 15 more multi's and Del Norte is done.


   
ReplyQuote
Holmwoody
(@holmwoody)
Eminent Member
Joined: 3 years ago
Posts: 37
 

@pbeprod Thank you for the update and very well done. It was a big library! I had one or two health issues this week so am now trying to complete the MM Guitars (1,2 & 12) today. Looking forward to the release of the Discover Station Orchestra library at the end of the month.


   
ReplyQuote
(@thsve)
Member
Joined: 3 years ago
Posts: 315
 

@getdunne

Hi Shane,

Sorry to bother you, but I am stuck 😳 .

I have got the pack/repack scripts integrated and working in my Python programs. Now I've started to se if I could rewrite them in C# (mostly for learning). I can't find any equivalent to the function: . to_bytes (4, byteorder = 'little'), in the repack script, in C#. Trust me, I've really Googled it.

Do you think you can point me in the right direction?  

Win 10 and 11/Cubase Pro 12/Unify/Wavelab 8/Vienna Pro 7/Spectrasonics all/NI 13 Ultimate/Izotope MPS2/Serum/Cthulhu/Scaler 2.5/MusicLab guitar vsts/BIAB 2022/TouchOSC/Metagrid Pro etc


   
ReplyQuote
(@getdunne)
Illustrious Member Admin
Joined: 3 years ago
Posts: 3907
 

@thsve

My knowledge of C# is quite limited, and I haven't seen the rest of your code, so I can't give you any actual code.

The magic word you need for your Google searches is "endian". Motorola CPUs used "big-endian" format, where the high-order byte comes first in memory, but this has been mostly superseded by "little-endian" format, where the low-order byte comes first in memory. Intel led this change, but these days it's pretty much universal. (ARM CPUs can be configured to use either format, but little-endian is more common.)

See e.g. https://stackoverflow.com/questions/19560436/bitwise-endian-swap-for-various-types


   
ReplyQuote
(@thsve)
Member
Joined: 3 years ago
Posts: 315
 

@getdunne

Thank you for taking time to answer this, I know that you are really busy, so just ignore this if you want.

I'm quite new to C# and struggling along with Mr Google 🤗. I first tried C++, but that is way over my head at the moment.

I am trying to "mimic" the python script with this code in C#:

using (BinaryWriter writer = new BinaryWriter(File.Open(unifyFile, FileMode.Create)))
{
byte[] strbytes = Encoding.ASCII.GetBytes("VC2!");
writer.Write(strbytes);
byte[] intBytes = BitConverter.GetBytes(newArray.Length);
writer.Write(intBytes);
writer.Write(newArray);
writer.Write((byte)0);
}

The file created is correct for all parts except for the "intbytes".

In the attached picture the first is showing the result with my C# code above. The second same file created with the Python script. 

I've tried a lot of different approaches found on stackoverflow, but no success so far. I will keep trying just for the sake of it 😎. 

 

 

Win 10 and 11/Cubase Pro 12/Unify/Wavelab 8/Vienna Pro 7/Spectrasonics all/NI 13 Ultimate/Izotope MPS2/Serum/Cthulhu/Scaler 2.5/MusicLab guitar vsts/BIAB 2022/TouchOSC/Metagrid Pro etc


   
ReplyQuote
(@getdunne)
Illustrious Member Admin
Joined: 3 years ago
Posts: 3907
 

@thsve

I don't know the details of these C# APIs you're using. My best guess is there is nothing wrong with your writer.Write(intBytes) call, and the problem is that BitConverter.GetBytes(newArray.Length) is returning more than four bytes. Try casting newArray.Length to a 32-bit type like UInt32; see https://docs.microsoft.com/en-us/dotnet/api/system.bitconverter.getbytes?view=net-5.0


   
ReplyQuote
(@ssquared)
Member
Joined: 3 years ago
Posts: 216
 

@thsve  C# automatically chooses endian based on the current architecture.  You can see if it is little endian by calling the static field IsLittleEndian on BitConverter (eg:  bool isLittleEndian = BitConverter.IsLittleEndian).  If it is false, then you need to manually reverse the bytes you received.  It seems to be writing other data without issues, so it seems OK.

Assuming newArray is a C# array, then Length should be a 32-bit value and return 4 bytes.  What type is newArray?

Some ideas:

  • First try to write the integer directly:  writer.Write(newArray.Length);
    • C# has nice features for writing data directly without first going to bytes.  To read, there are methods for ReadBoolean, ReadInt32, etc.
    • You can also try writing the string as writer.Write("VC2!");
  • Check the number of array elements in newBytes using the Debugger (or write it out to the console).
  • You can also verify intBytes has the right value by calling: int convertedLength = BitConverter.ToInt32(intBytes, 0);  0 says start at the beginning of the array.  See if convertedLength is the same as the length of newArray.

My guess is newArray is actually a different size in your C# code from Python.  For example, newArray might contain those three highlighted bytes (EF-BB-BF) following the 4 bytes for the length.  And this is why the bytes for length are different between the two (BA-3B-07-00 and DD-3A-07-00).  I'm not sure I did this right, but those values might be length 474,042 (C#) and length 473,821 (Python).

You should check the size/length of newArray in both C# and Python.  It  looks like you are writing XML.  Check how you are generating the XML between the two.  Maybe the C# is adding invisible characters (like a return and/or new newline character) where the Python does not.

Ableton Live 10, Omnisphere, Native Instruments (Pianos), Spire, Hammer + Waves, Heavyocity (Ascend and Mosaic Keys), Diva, SynthMaster, Alchemy 1.55, Addictive Keys, Unify


   
ReplyQuote
(@thsve)
Member
Joined: 3 years ago
Posts: 315
 

@ssquared

Thanks for your answer, it inspired me to troubleshoot a little wider. After a lot of sweat and tears and elimination of error causes, I managed to find the cause. When I save the updated xml file it was saved with BOM (3 bytes). So the problem wasn't in the repack function at all 😎.  After I removed them, the repacked UNIFY-file is correct so now I can continue developing with the program.

This clearly shows that I lack a lot of knowledge about this. I had no idea what BOM was and barely even now 😊 .

Win 10 and 11/Cubase Pro 12/Unify/Wavelab 8/Vienna Pro 7/Spectrasonics all/NI 13 Ultimate/Izotope MPS2/Serum/Cthulhu/Scaler 2.5/MusicLab guitar vsts/BIAB 2022/TouchOSC/Metagrid Pro etc


   
SSquared and getdunne reacted
ReplyQuote
(@ssquared)
Member
Joined: 3 years ago
Posts: 216
 

Fantastic!!!!  Great to hear.

Ableton Live 10, Omnisphere, Native Instruments (Pianos), Spire, Hammer + Waves, Heavyocity (Ascend and Mosaic Keys), Diva, SynthMaster, Alchemy 1.55, Addictive Keys, Unify


   
ReplyQuote
Share: