Unifying and Optimizing AECO Workflows with OpenUSD

VISHU BHOOSHAN: Hello, everyone. This is Vishu Bhooshan from Zaha Hadid Architects. I will be presenting about unifying workflows with open USD and how we are doing at the Zaha Hadid Architects. A bit about myself, On a day-to-day basis, or on a weekly basis, I do these four things, which is related to computation and design research, but also in dissemination of knowledge at these two affiliations, one at the computation and design group at Zaha Hadid Architects and also at the UCL Bartlett School of Architecture.

A bit of introduction about the computation and design group at Zaha Hadid Architects. It was started in 2007 by Patrik Schumacher, Shajay Bhooshan, and Nils Fischer. It was started as a project independent research group, with focus on geometry initially, and how we can adapt tools and technologies related to computational geometry processing and computational design into architectural projects.

So as you can see, when we started in 2007, we set out with small scale projects, where we were testing novel technologies in both design but also novel digital and manufacturing technologies to deliver those projects. As the research has matured both on the design side and on the delivery side, the scale of the projects, as you can see, is also scaled up to stadiums, to metro stations, to master plans, so on and so forth.

These are the current set of people in the team. All of us are trained as architects with interests in design technologies or spatial technologies, as we call it. Research typically follows this timeline in our team. It goes from being project independent research, as I said. Then we develop toolkits and specific applications based on that.

And this gets initially applied on pilots or special interest projects. Once it has been tested and made robust, then we deploy it on large scale applied with applied research stage. These are the current strands we are currently working on in terms of research, which is high-performance geometry, game technology, and participatory design, future-twins, and metaverse, and more recently, also looking at machine learning and AI.

This presentation will focus more on the first three aspects to delve a bit more, to understand a bit more on the machine learning and AI what we are doing. Please listening-- listen to the other session where I'll be talking about tectonism via AI. That one should also be available soon.

The agenda for the presentation would be as this, looking at what is spatial technology stack, and how it relates to open USD, how such a spatial technology stack, which we call ZSPACE, is collaborative, it's aligned to practice, and how it's enabling us to do online participation of stakeholders. And then eventually I will conclude the presentation with a few slides.

So ZSPACE, the spatial technology stack, is a software agnostic framework which we are developing in-house. Why we started developing that is because in AEC we believe that there is a streetlight effect when it comes to technologies. We tend to look for things only under our own streetlight. Although there are better things might be in other streetlights.

When I call streetlights, it's probably related to other industries. Why so? Because we believe architecture is a cultural production enabled by technology. This has been the case ever since a long time. And it's not nothing new.

People like Frei, OTL, and Antoni Gaudi. As you can see on the left, we are always blown away by the spatial and visual aspects of it. But all of it is actually supported by novel technology in both structure, fabrication, et cetera. And the same happens in movies as well, or animations.

It's also a cultural production, where we are always again, blown away by the visual aspects of it. But if you dig deeper, the technical aspects are also as novel and cutting-edge. So we tend to look at other industries to get our inspiration, especially while looking at technologies.

So what can AEC tech borrow from other industries, like gaming and animation and computer graphics, would be the computational geometry and the game tech aspect. So we specifically look at tools like Autodesk, Maya and 3ds Max to look at their digital content creation pipeline, which is mesh processing to create polygon models, texture packing, so on and so forth.

So what can we borrow from that is called something for computational geometry. We would be getting architectural geometry, which is a subset of computational geometry, which I'll get into detail a bit later. And also looking at the USD format used in Pixar and elsewhere to do the collaborative workflow.

So in the same tech, as previously mentioned in the previous slide, Pixar USD, we now can be-- it's much more accessible via the NVIDIA Create kit. But also a lot of softwares are now natively adapting this technology. So when we called spatial technology stack, which we call ZSPACE is a combination of these technologies.

It's called Architectural Geometry, AG, plus Computational Geometry, CG, and game technologies. So it's a convergence of all spatial design disciplines and technologies. The design paradigm of which we are-- which the group is part of and also in general, Zaha Hadid architects, is called tectonism.

It's a subsidiary style within the paradigm of parametricism. The main objective here is to make visible in shape and stylistically heighten structural fabrication, environmental, and spatial performance criterias. If tectonism is the design paradigm, architectural geometry is the technology paradigm which supports that.

So architectural geometry focuses on creating shapes that are structurally aligned and fabrication optimized. This was initially coined by Helmut Porttmann at TU Vienna. These objectives are further-- we have added on other performance criterias, like environmental performance, spatial performance, so on. So any performative aspects captured in the geometry is called-- for our specific-- for architecture, it's called architectural geometry.

The stack borrows from research in architectural geometry at various research institutes across the world, where we have active collaborations with some of them, like TU Vienna, Block Research Group, Interactive Geometry Lab at the ETH Zurich, et cetera.

Now, the guiding principles for us is that geometry becomes a mode of transfer between the various stakeholders because it is visual. If the geometry is not right, that means we need to fix some things. So that's why it also becomes a very effective tool of transfer between various stakeholders.

The main objective here is to learn from first principles and write our own set of tools so that we can combine multiple methods to a specific project as and when required, and build up tools which will aid in design, assisting in early-stage design rather than replacing of the designer. So another objective is to augment BIM.

So typically, BIM-- we can see of objectives on the right side, where we have traditional construction techniques, which are already embedded into BIM platforms. Now we want to incorporate using Open, use the other factors, like all the performance metrics beside how it's going to be fabricated, if it's going to be done with novel technologies like 3D printing, how are these information embedded into a file format, which becomes aggregated over many stakeholder collaborative setup.

Because we are a design office, so our tool sets are predominantly focused for early-stage design. But focus is both on content creation, which is concept stage and early stages of competition, but also develop tools for delivery of these projects for later stages, like schematic design and detail design.

As you can see on the left, on the content creation side, it looks at the geometry processing, looking at minimal surface, 3D graphic statics, 2D graphics statics, how to do things with robotic manufacture, 3D printing, et cetera, while on the right side for content delivery, we are also looking at planning and sequential, how we can embed early stage costing, et cetera.

And in the central part, you can also look at game tech and platform system design, which enables looking at configurators and enables participation of various other stakeholders. Our core framework, which is software agnostic, is written in C++, but it talks to the software stack we have in the office. Mainly we have Autodesk Maya. We have NVIDIA Omniverse, Pixar, Unreal, Rhino, and Revit.

And we have extensions to this core framework in Python and C#, which talks to the respective ecosystems. Looking at why we are doing it is because it enables collaborative workflows, what we call as integrated design to production pipelines. I'll explain this through a certain case study projects.

The first one is reactors, which was a collaboration between the Block Research Group at ETH Zurich, our team at Zaha Hadid Architects, Incremental3D, which was the robotic 3D printing company, and Holcim, who are looking at novel material, specifically for 3D printing. There are concrete-- cement company.

The objectives of the collaboration was to create high-performance shapes in-- using lightweight, low-energy, low-carbon materials, and to come up with a new language for concrete, to use enduring and established material, like concrete, combine it with ancient wisdom of masonry, and novel and digital robotic manufacture.

Another objective was to showcase sustainability of digital concrete, how it can be used-- reduce, reuse, and recycle, how this can be accentuated and highlighted in a-- first as a technology demonstrator, and then subsequently, we did a second part, which became-- it is now permanent.

So this was done in 2021 for the Venice Biennale. It was exhibited there. And it showcased the integrated design to production pipeline as seen in this video. So the right side is the USD file or the file formats, where, you can, at each step of it, you can add attributes.

So starting from the design skeletal graph to a graph mesh, to subdivision, and then our structural information such as truss network analysis and-- was added into the same file format. Once the blocks were done, which we are about 53, then the slicing of it into print paths was also embedded into the same file format. And the G-code, which was eventually used for the production, is also embedded in the same file format.

So as a designer or any stakeholder, looking at the file, we already have all the various information we need with respect to design, with respect to structure, with respect to fabrication, any change in any one of them is easily captured, and we can adapt. If there are clashes, we can adapt to various-- make changes such that we don't have those clashes.

And this enables very quick iterative iterations. So we were able to quickly iterate through multiple design shapes, whilst being aware of the structure and the fabrication constraints. So this enabled us to quickly create such a project in a timeline of four to six months.

And the print parts were generated and very quickly done in three months time. And then it was sent to Incremental3D, which were printing it in Austria. Of all the 53 blocks, once printed, were then shipped over to Venice for the Venice Biennale in 2021. As you can see, it comes in boats.

And each of the blocks were assembled on site. So it requires, as you can see, a scaffold in this case, because and this was something we subsequently improved upon how to minimize single-use scaffold, like the waffle system you're seeing here. So similar to any wall structures, you start from-- in this case, we were starting from the footing, and then going towards the center.

As you saw there, it stands in pure friction. To enhance the friction, a neoprene pad was used between blocks. Once all the blocks were done, now it's standing purely in compression. There's no reinforcement inside of it. So as you saw with the structural simulations, if there are about 70 people standing at a single point, this could collapse. But it is well and above the required structural performance.

And Striatus-- why it's called Striatus was to showcase the tectonic aspects which come from 3D printing, which is like striated lines. And we wanted to enhance those in the tectonic features of the design. Once that was done, a more permanent version was done at Holcim in Leon. So this was, of course, Striatus 2.0, and we call it Phoenix.

Why it's called Phoenix, because we wanted to highlight the circularity of such a system. That's it. If we wanted to use the same exact bridge, because of the material separation between concrete and steel footings which were used, it could be easily be disassembled and assembled elsewhere.

But we also wanted to showcase that the same material could also be used to do a different structure. So which was what we had done, in this case, was to recycle the material which was used in Striatus to create and adapt to a new shape, which incorporated advancements which we found in the-- found as issues in the first version.

So what we were trying to do was to make less dense infills. As you can see on the left, it's more triangulated and more number of-- more material was required, whilst on the right, we went with the more vertical bracing system so as to reduce the bracing material, but at the same time also reduce one-time use scaffold.

So you can see on the left it's more-- could see more of the timber. And on the right, you see less timber and more of the steel re-usable scaffolding system. Another aspect we developed was-- previously the blocks were-- the biggest block was about two meter in height, which was difficult to position it on site and also shipping. So we reduced the block sizes by half.

So if we had 53 blocks in the previous version, we had about 102 in the new one. But it enabled us to also reduce the print time because if there were issues, you didn't have to restart again after, especially if it was a large block. So because all of the blocks were smaller, each block took about an hour to print.

And in the assembly also, instead of assembling one block at a time, something called [? cassettes ?] were developed, where you assembled-- as you can see on the bottom-right image, six or seven blocks were assembled into what we call a [? cassette. ?] And this [? cassette ?] was then lifted and placed in position. So this also enabled it to be, the production, or the assembly process, to be sped up.

The material aspect, as previously mentioned, was using-- exactly recycling of the material from Striatus. It was grinded and recomposed, and that material was used for the new, more permanent version. And as you can see, the steel was also reduced. We went away from the steel footing previously seen in the Striatus one to a more concrete footing.

Moving on, we wanted to-- the previous set of projects was showcasing the technology demonstrator, where we are working with new advancements in technology, working with partners like ETH Zurich, or Incremental3D and Holcim. And the tools we developed through such technology demonstrators.

We also-- as it scales up, becomes aligned with practice. And then application in-- so this section will showcase how these toolkits are currently applied on large scale architectural projects. One of the first projects we want to showcase was the Mathematics Gallery at London.

Such a toolkit, which has been developed over a period of time, enables procedural design. So now we had experience with previous similar minimal surface. So now we are able to quickly use those tools developed in similar technology demonstrators, which enables to procedurally design shapes, but also with an awareness of fabrication.

So we were able to quickly iterate through a wide range of options whilst all of the-- whilst all of them were still amenable to fabrication and also structurally aware. What this kind of toolset enables is, as a designer, now you can-- you are also enabling negotiation with consultants.

In this case, consultants would be fabrication-related, and then you can come up with a participatory-- collaborative model, where the model is first analyzed by the fabricator, gives you a version of the same pattern. But because we had developed tools, we could also propose certain same patterns which are performing in terms of aesthetics as well. So as you can see, on the one to the right.

So there was a back and forth between designers and fabricators, and eventually, you settle upon something in between. Similarly, the museum also had a set of benches which was robotically wire cut. So this falls under the criteria-- [? geometry ?] falls under a criteria called rule surfaces.

So as long as the geometry is ruled, it's gone-- it's possible to be cut with the wire cutter. And those constraints were embedded early on into the design process. So the parametric tool, which was developed for each of these benches, has similar criteria, that it takes an input curve.

And because of the production strategy of doing wire cutting, we could do-- all the 16 benches in the gallery were unique, or customizable, as compared to if we went with traditional mold-based techniques, this would have not been possible. We had to optimize all of the geometries to two or three molds given the time constraints.

So what robotic artwork cutting enabled was that we were able to stretch out the design process a bit more. As long as we took into constraint the fabrication constraints in our design process, it was, the production, of it was much more faster than doing it via mold. So each of these blocks, as you can see, has two sets of formwork.

One is the negative and the other one is the positive. In between these two is a three-centimeter thick, high-performance concrete. So the inner one is a large formwork. So it also-- it's inside this, and it makes the whole of the benches very light.

The same thing, similarly, as-- can be seen applied towards larger scale projects, like in this case, the Xian football stadium, where we were involved with the project team to work on geometry rationalization of the roof, the cable net form finding, and also the lure system and the facades, et cetera.

Again, the tools which were developed through technology demonstrators, we have done previously. We were able to quickly-- initially, in the competition, or [? pre-SD ?] stages, we were able to quickly generate these kind of shapes, but also procedurally generate the cable net roof structure, which was, again, a back and forth with the engineers on the project.

What this enabled, again, was, a back and forth with the structural engineers, was to go from what we have on the negotiation in getting-- going away from the truss in the central in the competition stage to a much more lighter roof in the [? pre-SD ?] and subsequent construction stages. Multiple topological studies were also done to check which one was performing better. And the best one was chosen for further development.

What the file formats and USD file formats also enables is to-- the procedural aspects of the geometry creation can be easily embedded into the file format. This is what would be transferred to a fabricator so that they can recreate the geometry at their end very quickly or very easily, because all of the steps of the procedure is embedded into the file.

The history of all the geometry is created. And a step by step guide is given to them for each of the geometrical aspects we saw in the stadium, be it be this is showcasing the facade or the terracotta paneling. The same thing is done for the lure, same for the tiles on the roof, so on and so forth.

So all of these tools is now-- is embedded into what we call a ZSPACE kit, which is built with NVIDIA Omniverse kit app. So we build our own custom based on the repositories given by NVIDIA. It enables us to create a geometry collation platform and collaborative working. So where multiple people in the team are able to work in parallel whilst having an understanding of what the global thing is.

This enables us to create these workflows, where we can have people working in their preferred ecosystem or softwares, whilst all of them, every couple of days, push these geometries to a master USD file, where as project managers, we're able to quickly go into it and look at any issues there are. So it enables us to collate the geometry in one place and quickly see the updates as and when they are made.

But it also enables us to work in parallel. A lot of people, like if the designers are changing the geometry, people who are working on the geometry rationalization are using a base USD to work on top of it. And if the base USD updates, then the optimisation procedures also update. And this kind of-- as you can see, all of them are USD based.

It takes a while to set up the template. But once you have the template, this can be easily used across multiple scale projects varying in scale, from small scale hydrogen power stations to large scale infrastructure projects to cultural buildings to master plans, et cetera. And as you can see, the same template file structure was used across all of these projects.

And again, you can see there's-- the office uses multiple softwares across the projects, including Autodesk Maya, Rhino, Unreal Engine for rendering, so on and so forth. So next product is the USB connectors, which enable these connections to the various platforms.

So we are-- here, we are looking at what are the native connectors which are available as Autodesk Maya has its own native connector to USD, which we are taking advantage of, also the API of it. And where it is not available, so we have our own viewer, which also enables us to do dashboard toolkits and toolsets.

So we built our own custom connector to the USD using, again, NVIDIA's client API. So once that is done, we could also enable some features which are missing in other connectors so that we can add attributes. We can keep a track of all the things we previously mentioned, like the fabrication, the structural aspects, as custom attributes into a USD file.

And all of those have been also been set up into custom API calls such that it can be easily done in environment of choice. Now this showcases how the API of Maya and Omniverse was used to develop these connectors across various platforms like Maya, Rhino, and C#, et cetera.

This is a video which showcases how this works. So an input geometry is taken, and then-- from a choice of the designer, and then that gets exported. And so this showcases how multiplatforms can also be used. So here we are connecting into our nucleus server to pick up the geometry, in this case, looking at a standalone platform.

And here, using-- once we read the USD, we are able to run the tools we have set up previously, which is to, in this case, optimizing of each of the roof panels for planarity. So if it is green, that means they are planar. So as you can see, over a period of time that most of the panels become green or planarized.

And once this is done, you can then export it back to whichever platform of choice, whether it is Maya, Rhino, Revit, et cetera. And in this case, it's showcasing the import back into Rhino. What it also, these connections, enable is we are able to capture the same simulations into a USD file as an animation so we don't have to run it.

So we, generally, saw-- we generally create a lot of animation content as well. So the simulation itself is translated into an animation file so we don't have to set up these keyframes again. So this is also an added advantage of creating your own custom USD connections. It also enables you to visualize various data visualizations.

In this case, you can also look if things are planar or not, if they are printable or not. What's the time to be printing? All of these added metadata can be easily be visualized in using a USD file. It's easy to add these attributes, but also to visualize them in any native platform of use.

And the same could be applied for other projects scales as well, small scale technology demonstrators to master plans, et cetera. And you're also enabled-- so this was for urban design project. So you're also able to create these layers of structure and call them as in when needed, and visualized to showcase various data aspects of a project.

This part is showcasing how we are accelerating computation using GPU. So we saw the 3D-printed bridge, where we were using signed distance field to create these print paths. And typically, signed distance field is very time consuming. But in this case, we were able to embed it inside of GPU computing using NVIDIA Cuda, which enabled us to do this calculation in real time and very quickly.

So we were able to generate-- if a block was taking about 120 seconds on the CPU, it takes like one second on the GPU to run it, or 1.5 seconds. So this enables for quick iterations of the print-path test as well. So this is, again, a back and forth between designer and the fabricator to get the one single print path and getting all the parameters correct.

So in this case, there were 102 blocks, all of which could be generated within two minutes. Similarly, we are also capturing now developing tools related to solar. This is what we call Maya Solar, where the designer is able to-- while they are making changes to the input geometry, they are getting real-time feedback on the solar radiation.

And this showcases that you could do, in almost real time, the-- while you are designing, you're having an understanding of the environmental performances, in this case, looking at solar radiation. But you could also do the same for hotspot analysis. Similarly, especially when you have convex geometry, we'll have to check if there are points of aggregation of light concentrations which we want to avoid.

So these could be quickly visualized very early on so as to avoid such situations so we don't have to rationalize it at a later stage, but can be visualized and tackled at early stage, or conceptual design stages. Same thing could be done for the shadow analysis and occlusion, also, to understand which are in shadow, and if those needs to be addressed in terms of daylight hours, et cetera.

So all of these toolkits, which are built on top of NVIDIA's [? Create ?] is enabling us to have more and more overlaps between linear scopes, such as architectural geometry, engineering and structural design and fabrication detailing. And what it's enabling is that each of these individual tiles, because of these overlaps, can be slightly longer so as to iterate through multiple options and thereby optimizing it.

But at the same time, globally, you're also saving time because you have these overlaps. Even if it is 5% to 10% of overlaps, that's already a big gain in the global timeline of a project. And more importantly, such pipelines are also enabling visualization-- architectural visualizations, to be embedded very early on, where you don't need to set up-- you don't have to wait for the final geometry to start setting up visualization.

You can start setting up from very early on because of the layered structures inside of USD. So if the geometry is at the base layer, and you're making changes on top of it, it's still-- if the base geometry changes, you still-- the above changes still will be valid. And so you can work in parallel since the beginning.

Other advantages of such a spatial technology stack is that it enables participation. When we say participation, it's like stakeholder participation, but it also online compatible. So you can push the technologies like Pixels-- in combination with technologies like pixel streaming, or GDN, we are able to also plug it into the web.

So why are we doing it? We wanted to-- we are developing configurators, where we are designing for choice, where it is system design for architects, engineers, fabricators, and developers. So it has, again, architectural-geometry related things, where we saw previously how we collaboratively we work with all of the stakeholders to produce a collaborative geometry.

And the same thing could be done into individual tile sets, thereby creating a-- enabling for participation online, where you could quickly, as a end user, be able to quickly generate these variations. Even if you had five types of walls, five types of balconies, five types of roofs, there's a wide range of combinatorics which can be explored.

And this is what we wanted to crowdsource, put it online, so that we can quickly generate multiple sets of various geometries, in this case, residential units. Once you had the geometries, it could be-- these tile sets can be easily put onto the web.

In this case, using pixel streaming, where it was built into a configurator where the end user could identify where they wanted their house, what kind of unit they wanted. And they could customize the interior, the exterior, based on the available kit of parts they had. And the visualization aspects are also getting improved on a day-by-day basis.

So the user or the end user is also able to walk through the spaces they design and see for themselves how it feels to be in that neighborhood. It's also leading to a platform of these web-enabled things, where your paradigm of design once, and use twice. So something called as Future Twins, you design once, to-- use the same set of tools to deliver the project physically, but the same set of tools could be used to optimize for virtual use, people, like putting it into a game environment or into the metaverse.

So this is showcasing the application of how-- the file format, we don't have to optimize for two different use cases. The same set of optimization and same set of tool toolkits work both for the physical and the digital world. It's also enabling us to create these games.

This was developed with Unreal for Fortnite, and it's called Merlin, where, again, the same set of tool-- same strategy of using tiles can be embedded into game systems, where we can crowdsource the creation of these architectural buildings in a gaming system and enables us to create a lot of data sets of building types of various typologies, like residential buildings, office buildings, landscape, et cetera.

And once you have such a well-curated set of data sets, it could also be-- it also connects to spatial models of learning from these geometries. So it becomes a data set from which you can learn the procedure of the combinations, but also the geometry itself.

And that is something we are currently exploring, how this could be learned, so that we can create these spatial data sets which are currently crowdsourced, how it could be quickly generated using, or assisted by, AI To generate the spatial models. So in conclusion, through the presentation, we showcased how spatial technology stack powered by open USD is scientific exciting and fun, but at the same time economical and profitable and productive.

It is entrepreneur friendly as you can collaborate with a lot of new startups which are looking at these novel technologies. It's a historically continuous, as we saw with wisdom of ancient masonry, looking at the advances made in computer geometry and computational geometry processing, how that could be embedded into architectural systems.

And it is also socially adaptive because now you can enable-- add it onto the web and get more and more people engage on the content digitally, but also performative sense in terms of spatial aspects of how social it is. That could also be a performance criteria you optimize for.

And the next immediate outlook for it is that it is-- such spatial technologies are being accelerated as more and more people are starting to adopt it, especially the pipelines using USD, where startups are becoming scaled businesses. So where we collaborated with them when they were startups, now they've become bigger businesses. So it's easy for us to communicate with such companies.

And more so now, a lot of the design companies like ourselves are also building their own prototype labs inside of their companies so as to test out these tech-- so as to test out these technologies in-house. And that's enabling the adaptation of the technology stack, heightening adaptation much of the technology stack.

It's also being accelerated by alliance of open USD, which is headed by Pixar, Apple, Autodesk, and Adobe, and NVIDIA. But a lot of other companies are also joining in and adding technologies which are-- others can use, and hence enabling us to quickly adapt to such novel technologies. It's also being accelerated by integrations with AI and collaborative platforms.

So here you can download a lot of open-source AI models currently, and quickly embed it into design platforms so we can understand and check how this could be used in a design setting. And the same is the case for the collaborative platform, which was showcased in the presentation.

And being accelerated through these kind of crowdsourced embedding into web so a lot more people can contribute towards it, and a lot more people can engage with it. And this is enabling something known-- a paradigm known as geometry for all.

And that enables us to speed up the process, again, more and more people and the computation aspect is also getting accelerated because of these things that it needs to work on the web. So please join us. Let's join and collaborate to create to-- disrupt the industry. And that would be my time here. Thank you. Thanks a lot.