WWDC 2018: USDZ file format

WWDC 2018: USDZ file format

I'm not sure what I should think about the USDZ file format yet. Especially since it mainly is a zipped USD file which already exists since quite some time. USD is a scene file format which was intended for big Hollywood productions. So on first sight that file formats seems to be total overkill for something as simple as AR.

But now to the good informations. USDZ will be accessible via the SceneKit and Model I/O frameworks. So I can use these frameworks to import / export USDZ files. That makes things much easier. Especially since Cheetah3D already has a SceneKit file loader.

The fact that USDZ uses PBR metalness/roughness shaders also fits very well into my current development of the upcoming PBR shader support in Cheetah3D 7.3.

So supporting USDZ shouldn't be too difficult. Now I just have to find some time to start work on it. That's actually the most difficult part. :wink:

P.S. There is one thing that actually made me pretty angry about Apples USDZ plans. And that is the fact that Apple informed big companies like Adobe much in advance about their USDZ plans while indie developers like me have to watch the keynote to hear about them. But that's Apple, nepotism at it's best.

Update (June 28.): Please also see the following posting
https://www.cheetah3d.com/forum/showpost.php?p=112592&postcount=12
 

Attachments

  • Cheetah3DScreenSnapz002.jpg
    Cheetah3DScreenSnapz002.jpg
    342.4 KB · Views: 1,336
Last edited:
Apple often gives strategic developers advance info on releases, especially if they want some kind of demo on stage. Not much you can do about it…
 
Not much you can do about it…

Yes, nevertheless annoying.:wink:

Just for curiosity does anybody use SceneKit or ARKit at all? Since the USDZ file format is mainly intended for usage with these frameworks.

I have the impression there is not too much interest in them.
 
No, I don't use anything like that.

But somehow I get a gut feeling that Apple is going 3d with some app, like they did with music with Logic Pro some years ago. It's somehow the thing that's missing. So there could be a bad surprise for you in waiting.

And I understand fully well that you're pissed off by this apple's behavior. I'm no developer and I'm pissed off as I see a lot of diversity going down the thing. In 3d I just can't see struggling firms like Newtek (well, they don't struggle, but lightwave does) and others to commit to this work intensive change. But it's not just the small 3d market. At least that could be a small benefit for you with Cheetah.

If they really try to force every developer to use their store, remembering me somehow of Marlon Brando as Godfather, that seems to me, as if they just want a cut in. So more of smaller and middle developers will Mac let be. And with it some software I use (as much as I like the store).

So it's in the end not only a problem for developers, it's one for the users too. They will get a smaller world appwise, which is something I don't like.

I left the windows environment two years ago because I was one of the few who experienced the nasty way of being forced to update to win 10 (and with a small almost not killable process that fucked up some of my drivers, meaning I had quiet a few weeks of repairing a system daily till I got rid of that thing without it coming back). Windows 10, even if you can get rid off all the spy processes and unnecessary apps, is still a no go for me.

On the other hand, in all probability, some of the apps I use for my work could not be available anymore in Mac in a few years. Others are Mac only and not available for anything else. And Linux, well, there is too much missing I'd need.

So, in the end, it's the same situation when I first heard about win 10. Wait, if it will be as bad as feared, and hope it will not ...
 
Just for curiosity does anybody use SceneKit or ARKit at all? Since the USDZ file format is mainly intended for usage with these frameworks.

I'm using C3D heavily to create models to use in my iOS AR app I have developed.
Up to version 7.2 I have exported my models to dae files and using a custom written SceneKit editor/mapper to create PBR materials and to optimise my C3D models to create working SceneKit files without fiddling every change in the model using Xcode's SceneKit editor.

So I'm happy to see that with version 7.3 beta 1 the existence of a PBR shader. Now my SceneKit tool can import your glTF files.

I'm using a lot of variants for my materials to choose in the AR app. AFAIK USDZ supports variants and I will be happy if you will find the time for implementation.

BTW: Thank you very much for all your great work on C3D 7.x :icon_thumbup:
 
Apple is shifting as rapidly as they can to 3D.

We can see it in the push to Metal.
We can see it in the AR/VR push.
We can see it in the eGPU push.
We can see it in the new 3D file format too.
We can see it in their new microLED display technology push.
Oh.. you didn't know about that one? There have been articles on it but Apple hasn't been talking about it.

Get ready. I believe everything I have listed has been in preparation for what they can do with those new displays.

My Theory:
With a SUPER HIGH RES AR display (think 8K), why would you want/need a monitor anymore ?!?

Think about it. The entire display system for ALL of Apple's products becomes 3D AR/VR. With AR/VR, the resolution of your "display area" becomes at least 10x the resolution of your display.

The "device" just DISAPPEARS !! There will be no (eventually) watch, phone, laptop, desktop, Apple TV. Your "device" is just there, wirelessly. Put on your AR glasses and the entire world around you becomes your computing space.

Hmmm.. lets see. What would Apple need to implement this...
-A new high performance graphics engine (Metal)
-An AR/VR API
-An AR/VR capable file format
-A flexible way to upgrade any Mac to faster graphics (eGPU)
-and a super high resolution display tech that they hold the patents on !

These displays will be ready by the end of the year (at the earliest).
What do YOU think will happen once they ship???

(...and this doesn't even consider their custom ARM processors and GPUs. Both of which will bring this whole vision into a small performant package that can go with you anywhere)

http://bgr.com/2018/04/03/apple-watch-2018-specs-micro-led-displays/


Martin, I vote that you move to Metal sooner than later.

Jason
 
Last edited:
Hi Martin,

I’m currently working with ARkit & scenekit.

In my case the reason I bought a Cheetah 3D license a few months ago... as Cheetah 3D was a great way to build key animations for scenekit/Arkit.

It would be fantastic if Cheetah 3D supported Apple’s new USDZ file format and incorporates Physically Based renderers.

I have already been using Apples usdz converter which has been bundled with the latest Xcode 10 beta. It converts OBJ to USDZ fairly easily.

This command line converts a OBJ file format PBR to USDZ.

HTML:
xcrun usdz_converter Wineglass.obj Wineglass.usdz 
- g WineGlassMesh
-color_map WineGlass_Albedo.png
-metallic_map WineGlass_Metallic.png
-roughness_map WineGlass_Roughness.png
-normal_map .  WineGlass_Normal.png
-emissive_map  WineGlass_Emissive.png

Personally I think its going to be massive new 3D market opening up over time... particularly Apple’s “Quick look” which works straight from the Safari browser to upload 3D content into Apple mobile devices (with iOS 12 beta). Google Chrome will mostly likely support the format eventually you would reasonably assume.

Apple’s Quick Look gallery => https://developer.apple.com/arkit/gallery/

Surely thousands of e-commerce websites would adopt “Quick Look” as a way to sell or promote their product listings. Adobe has obivously got on board because they already see huge market potential... particular if they already know Google Chrome plans to follow suit.
 
Last edited:
Here’s example of USDZ file using the “Quick Look” running iOS 12 on Safari browser. The render on the display was produced in Cheetah3d. I exported obj file and converted the file to USDZ using the usd_converter command line tool bundled with Xcode 10. It renders great in the AR view.


example.jpg


close up...shows great detail.

example_2.jpg
 
I don't currently use ARKit or SceneKit but it's definitely something I've experimented with in the past and would like to in the future "when time permits".
 
Here's another example of USDZ format using the new iOS12 ARKit framework. I built the magnifying glass in C3D & exported to OBJ format.

To create the USDZ I'm running the following script to make the the PBR textures.

Code:
#!/bin/bash   
echo running usdz_converter
xcrun usdz_converter magnifying_glass.obj -g frame 
magnifying_glass.usdz -color_map gold-basecolor.png -normal_map gold-normal.png -roughness_map gold-roughness.png 
-metallic_map gold-metallic.png  -g handle 
magnifying_glass.usdz -color_map gold-basecolor.png -normal_map gold-normal.png -roughness_map gold-roughness.png 
-metallic_map gold-metallic.png -g grip 
-color_map gold-basecolor.png -normal_map handle-normal.png -roughness_map handle-roughness.png 
-metallic_map gold-metallic.png -g lens 
-color_map lens_basecolor.png -roughness_map lens_roughness.png -metallic_map gold-metallic.png -v

As you can see not the easiest setup, because you have to match all the different PBR images to the group for each sub-mesh. I found it easier to remember them by saving them as shell scripts so its easy to run them at a later date if I make changes.

But if C3D could support USDZ format natively... that would make things are so much easier.

you can view& download the file at https://www.fusionar.app/usdz/magnifying_glass.usdz

The file is 3.3 MB in size.

to read the contents of the USDZ file just change the usdz to zip... and extract it. That will give you the USDC file and the PBR texture files.

mag_glass
 
Thanks for all the feedback. I've already started experimenting with the USDZ file format. I currently plan to use the SceneKit framework but USDZ support in SceneKit is still pretty rough. But it looks like Apple is actively working on it since Beta 2 of Mojave already fixed various bugs I encountered.

Beside the problems in SceneKit the main problem is my lack of time. I still have to finish glTF support which I've already started. And I also have to add support for the dark mode. Not to talk about the sword of Damocles called "deprecated OpenGL".

So although USDZ support theoretically doesn't make too much work I'm not that sure when I will find the time to work on it.

P.S. One of Apples sample USDZ files imported into Cheetah3D. Textures don't work yet.:redface:
 

Attachments

  • Bildschirmfoto 2018-06-27 um 14.28.00.jpg
    Bildschirmfoto 2018-06-27 um 14.28.00.jpg
    75.5 KB · Views: 899
Hi Martin,

Thanks for the update.

Would a future C3D USDZ exporter include support for animation?

Just on that.... to-date I haven't been able to find any concrete examples for even creating a USDZ animation file via the xcode usdz_converter.

Unfortunately, Apple did not include any animation example in their online gallery (seems deliberate) even though the animation feature got announced at the the Apple WWDC :frown: & the other likely source Pixar only has one example of creating an animation via the open sourced USDA format, which is hard to test unless you install their open source USD pipeline and utilise usdview to play back the animation. However, even if you get the USDA animation working in usdview (and that is a big but), there is no concrete example of converting this to USDZ.

With these difficulties in mind, a developer on github has already attempting to hack the USDA format "Proof of concept of converting glTF to USDZ for AR Quick Look (iOS 12+)." its still early days as the usd pipeline on the mac platform is experimental, and even this developer has had no success installing it without errors!

It is puzzling to me...why Apple would announce support for animation in USDZ, with no supporting docs & no working example. My only conclusion is they want to give their partners Adobe & project Aero the first opportunity to market.
 
BTW I got the gramophone.usdz to load into C3D with textures by the followingm method:

1. change the suffix to gramophone.usdz -> zip.
2. uncompress the file.
3. click on gramophone.usda file and load into xcode beta 10
4. export the file as a DAE file.
5. import the DAE file into C3D
6. set all the polygon meshes to UV SET: UV2

If I exported the file to scn in xcode beta 10 and loaded into C3D.... I couldn't find the polygon or mesh... don't know if was a scaling problem??
 

Attachments

  • gramophone.png
    gramophone.png
    252.4 KB · Views: 735
  • retroTv.png
    retroTv.png
    384.2 KB · Views: 894
Hi,
the import is actually less the problem. It already works quite well. But so far it's not possible to save to the USDZ file format using the SceneKit framework. At least I wasn't able to achieve that and other users in the web discovered the same.

But I'm quite optimistic that Apple will add that in a future beta. USDZ support would be pretty incomplete without it.

Bye
Martin
 
Just update on my own exploration around USDZ format in recent weeks.

Based on my own testing and recent communication by the USD pixar team.

These are the means by which you can currently generate varying types of USDZ files

OBJ -> USDZ (static meshes, no animation)

ABC (Alembic) -> USDZ “currently” supports tranform animation (scale, position, rotation) only.

I’ve tested the Blender Alembic exporter and it does work with tweaks. You need to make sure that you have you polygons under parent groups/folder.. otherwise transform animations won’t work. There is also an compatability issue with the Alembic exporter using frames whilst USD interpretes timescales in seconds. A current work around is to edit the ascii usda file manually and fine-tune the time-scale. I can provided how-to instructions if C3D users are interested in details.

Bone Skeletal animations -> USDZ are currently generated via the Maya full version + USD plugin provided by Pixar. Given a full license cost of Maya is $3500, I’m hoping Apple is working on ways to provide lower cost alternatives.

I’ve uploaded a few examples of USDZ tranform animation to my website https://www.fusionar.app

Alternatively if you don’t have ios12 beta installed yet... you can check out the youtube video I posted https://m.youtube.com/watch?v=whPmjHA_1Ws

Martin, it would be great if C3D had full support for USDZ in the future, but I’m guessing without Apple adding SCN -> USDZ support its probably not on the cards in the foreseeable future.
 
Update, after spending many, many weeks trying to solve this. I finally found a reliable way to create usdz bone animations files from Cheetah built models. There is a several steps at the moment in relation to using command line tools & blender to make the final conversion to usdz... however, its doable. I've attached the Alien Walk model for people to play with for when IOS 12 is readily available in the next week.

I've attached the Alien_Walk as an example for anyone who wants to check it out.

AlienWalk.gif
 

Attachments

  • AlienWalk.usdz.zip
    106.6 KB · Views: 369
Last edited:
That's awesome. Could you share this process with us?

Ideally, we can create a Cheetah 3D script to export this right from Cheetah 3D.
 
Yes, nevertheless annoying.:wink:

Just for curiosity does anybody use SceneKit or ARKit at all? Since the USDZ file format is mainly intended for usage with these frameworks.

I have the impression there is not too much interest in them.
Yeah, I've used Cheetah 3D and would export to DAE and import these into Xcode and display them on an iPhone or iPad, this flow worked real well with the prior versions of Cheetah 3D. Apple however has gone to USDZ format now which Cheetah 3D doesn't support yet. I would definitely use this conversion mode if Cheetah 3D had a way to convert to USDZ since I have a lot of 3D in Cheetah's format. Right now Apple has Reality Composer to create the 3D content and it's in beta and quite buggy, plus there's no good converters out there to convert from OBJ or DAE into USDZ. If Cheetah 3D supported JAZ to USDZ I think they could quickly own this market segment.
 
Back
Top