Cheetah to USDZ for AR Tips

jasperpotts

New member
I have just spent 2 days fighting to get a Cheetah model to work in AR across a range of iOS devices. I thought I would share what I learnt to help others :) The main path I was going down was export as GLTF then use Apples "Reality Converter" this worked but the resulting files crashed on some iOS devices. In particular older devices. The final answer is in two parts:

Textures
Along the way I found that using PBR shaders got you much closer results in the exported file. For the properties like roughness or metallic, if I did not have a texture image then I created a set of tiny PNG files for Black, White and Grey and use those rather than the color picker. The color picker colors seemed to be lost in conversion.

Conversion
I started with trying all the Apple converters but they all gave the same results of works on some devices and crashes on others. The file sizes were 65Mb GLTF to 45Mb USDZ. I then found the Google open source converter USD_FROM_GLTF from https://github.com/google/usd_from_gltf. It was only available in source and a right pain to compile. But once I got it working it was amazing, took the 65Mb GLTF and produced a 23Mb USDZ that works on all devices :) there was some texture quality loss but I think that can be tuned with command line arguments. I was happy with the quality size tradeoff. Also the google converter is maybe 100x faster than Apples. In case it is useful my notes on how to compile are below.

Bash:
brew update
brew install boost
mkdir /opt/local
mkdir /opt/local/USD
mkdir /opt/local/usd_from_gltf
git clone https://github.com/PixarAnimationStudios/USD.git
cd USD
python3 build_scripts/build_usd.py --no-python /opt/local/USD
cd ..
git clone https://github.com/google/usd_from_gltf.git
cd usd_from_gltf
python3 tools/ufginstall/ufginstall.py /opt/local/usd_from_gltf /opt/local/USD --testdata
echo 'export PATH="/opt/local/usd_from_gltf/bin:$PATH"' >> ~/.zshrc

What would be AMAZING! is if Cheetah could build the Google converter in and offer USDZ export, worth a try asking ;)

The same model was used in a Unity app and as USDZ QuickView, the unity app can be seen at

Here is a screenshot of the AR Quickview USDZ

ARTruck.jpg
 
I would love to have built-in support for USDZ, especially since Apple's own stuff like SceneKit and Motion uses USDZ.
 
Last edited:
USD, or in it's compressed form USDZ, developped by pixar, is the one format that finally really has the potential of becoming the industrial standard, far more than gltf, collada and fbx (they all have their massive short-comings). I'm sure, it will have it's short-comings in the way materials are (or will be) translated in different 3d packages, or with light systems just not being the same, so there still will be enough differences in the end result (probably less when you use pbr). Another advantage is simply, it's not coming from Autodesk, but from a firm which isn't interested in selling software. Their main goal is simply put the same we want: A format that allows working with different apps on the same file without the hassle of conversions.

As more and more 3d developers see the advantage, it's not only good for AR-stuff but for everything that demands 3d data shoved around between different apps. Actually I don't just think it would be good to have a good support for usd (and usdz), but, in the long run, almost mandatory, so Cheetah can play nicely with all kinds of other apps. I hope, Martin soon has the time, to implement this.
 
Hi Hasdrubal. What specific shortfalls does gLTF have compared to USDZ? I ask because I have primarily been using gLTF as that is what most apps I use support but have been interested in starting to learn more and experiment with USDZ.
 
Hi Pegot

Actually the "short-comings" I mentioned have to do with my POV, which is centered more on DCC (i.e. shoving data around) than on putting some 3d file into a website or fb, therefore a bit one-sided and not truly fair. Additionally, take my opinion with a grain of salt, as I'm no specialist in these things and my knowledge doesn't go very deep.

For some use cases glTF is better suited than usd, because it does "optimize" the data, so in short, it's tailored for "real-time" use on a gpu. So it doesn't support a lot of shader attributes (I think, displacement is one of them), and you get only the same material using pbr-files.

That Apple chose USDZ and not glTF for AR was discussed as glTF seems better suited for this because it's faster. So it's probably a bit of a gamble on Apple's part which has to do with their new computer architecture. They seem pretty confident that the actual advantage of glTF (faster) can be neglected in the future. Not at the moment, though, so Apple doesn't convert everything into USDZ what the Pixar USDZ material actually allows. I read that there are especially differences in the material. So USDZ comes already in two flavors (probably Apple will sooner or later cancel the limitations, at least some of them), Apple's and Pixar's (Apple's was developed with Pixar, I believe).

USD was developed by Pixar for their own use to exchange data between their different software tools, some of their own development but also standard tools like Maya, Modo and Blender. They just don't care for real-time-view or optimization for gpus. It's really only about data exchange in full scenes with as much data as possible. I've seen it described as "professional 3d" data exchange format, especially with larger scenes in mind; which probably hits the mark. I actually never really tried it out (I have only one app that can handle it), and it certainly has some shortfalls of it's own, but it would be nice to get finally a format that really included all the necessary data for working on a file in different apps and rendering in still another without the necessity to adapt all the materials and keep the whole thing in subdiv. Or you can download a model (free or paid) and use it in any 3d app you want without all the hassle you have to go through to adapt the different materials or tripled polys etc.

So, it depends on your intended use what format is actually better suited. And if you start with something created in Cheetah and use glTF to export, there's not much of a benefit to use usd later on as "the damage" is already done (meaning the optimization). As always, the usefulnes of usd is / will be determined through how well it's implemented in the different apps.

Both formats have the big advantage of being open source, free to use for anyone, which isn't true of fbx which is a proprietary format by autodesk. They can change it as much as they like, and I believe you have to pay them for a license if you use it in your app. So it would be a good thing anyway to get rid of that, something that glTF couldn't achieve. But with usd it finally should be possible. (I myself use much to often obj).

Hope that helps.
 
Thanks for the detailed reply. Very interesting. Seems USD is more an interchange format while gLTF final optimized delivery.
 
I also agree that gltf is not a bad format and it was also just recently added to Cheetah3D.

The thing with USDZ is, however, that it seems to be the Apple thing and with Cheetah 3D being a Mac only tool, it seems odd to me that I can't export something that I can import into Motion, for example.
 
Thanks for the detailed reply. Very interesting. Seems USD is more an interchange format while gLTF final optimized delivery.
In a nutshell, but instant delivery where quality isn't the main thing.

In my post above I forgot to link to pixar's description of USD (the texts are several years old): https://graphics.pixar.com/usd/overview.html. Especially interesting are the workflow studies which show the potential. (a bit more technical and less easy to find is this https://graphics.pixar.com/usd/docs/api/index.html. It's not the same as docs on the other site ...).

Cheetah will not be all too often a part of a professional workflow or so. But there is what Somian said, at least for Apple's VR it should be able to export to USDZ. Also I hope that sooner or later for sites where you can download models etc. USD and USDZ (in this case not only a subset) will be the standard (on the other hand it's probably more work to include it in Cheetah than gltf ever was).

And no problem at all (I was aware of USD because of another app).
 
Back
Top