Virtools

Anything related to the authoring system Virtools DEV from Virtools (http://www.virtools.com).

Last time I gave an overview on what VBuilder is and how it improves iteration times. This time I'll give a small example by showing some extracts of what it takes to setup a project.

VBuilder is driven by commands coming from an XML file. Here is an how this can look like:

<!– register data resource –>
<register_dataresource xsrc="demoproject.rsc"/>

<!– load the cmo –>
<load_cmo xsrc="../CMOs/main.cmo"/>

<!– execute build process –>
<execute_script name="INIT"/>
<execute_script name="Load Buildings"/>
<execute_script name="Load People"/>

<!– set the save options for images and sounds –>
<set_save_options name="images" value="IMAGEFORMAT"/>

<– save to VMO –>
<save_changes name="content/demo.vmo"/>

So first one or more data resources are registered. Basically it's adding the search paths for external resources like textures and audio. Then a CMO is loaded. It contains the import and processing logic which then gets triggered. Usually you have textures outside but if you want to include them for the final file, this can be done like shown above with the save_options commands. At the end the new VMO, that includes freshly imported assets, is saved and can be started ie.e with the webplayer.

Btw. VBuilder also supports commands like setting values for script-parameters – very useful for making sure that debug-information is turned off for release builds!

So how does the assets import looks like? Well, this is adjustable to your needs but here is how I often do it:

So basically it's a scriptable build pipeline. Some people asked in the past years what we do and here you got it finally unveiled. It's actually a pretty simple concept and similar to bigger gamedev projects. I could bring it even closer to traditional gamedev projects: a future version could detect what files changed and only process those. That could be VBuilder V2.

Feel free to contact me about it, if you're interested to use these tools. I say tools, because I would add BBs like the Directory-Parser which is especially useful for projects that did not license the XE player.

In early 2006 I started to rethink the build-process for 3DVIA Virtools projects. It's a topic where many people from the Virtools community dive into one day or the other.

The problem

The standard method of doing an assets update for your project is, to let the 3d artist export the new assets versions, then a Virtools developer drops them into the CMO from the resource database and chooses what to replace and what to keep. One by one….

Some developer used custom scripted loading processes and so did we. We used text files and assets loading buttons for some projects. Text contained data of what material or mesh uses what shader and what attributes. But it came out that updating/maintaining those text files was quite error-prone and somehow not really artists friendly. Moreover building the final product for testing still required a bit of the time of a Virtools developer … time that was better spend on bug-fixing or adding features.

The solution

So, the new iteration was a command-line tool we created. Currently I am thinking about a new, better name. For now I will refer to it as "VBuilder". It's a one-click processor that is able to produce VMOs from CMOs using a list of simple processing-commands inside a XML file. A different XML file can be passed to the tool via the arguments, so it's possible batch-build for different VMOs.

We started to use directories as rough top-level for classifications of 3D assets: roads, buildings, cars, people. Adding a new car or a new building became a simple task: the 3D artist exports his new car as NMO to the "cars" directory, then he triggers VBuilder and within minutes a deliverable VMO file is available for testing.

What it requires though, is some up-front import-pipeline coding inside Virtools for each project (which might be reusable for different projects) and – in our case – naming conventions for objects and materials inside 3ds max etc.

What effect did we achieve with this?

Initially we had an iteration time of aprox. one day. This has to do with the fact that a Virtools programmer is not always available or ready to interrupt his development work for an assets integration-cycle.

So the artist may come to him and say: "Hi, I updated some buildings and some cars, can we have a look at it?". The programmer responds: "Let me just finished this little script first. It will only take 20 minutes" … … … then 2 hours later: "Ok, took me longer but now I am read… ???!! Hello?" – the artist went home as he comes to the office a lot earlier than the programmer!! He then does the integration and writes a notification about it. Next day, similar procedure. You see, this scenario really may happen and if you have a lot more Gfx ppl than coders it might get worse. I remember one project, where we had a junior Virtools developer doing mostly one thing: reimport new iterations from the graphics team.

With VBuilder the iteration cycle, for already available content-types, went from worst-case one-day to worst-case 5 minutes. Sometimes it only took 1 minute including startup of the player for testing. The best thing is, it even scales. I was able to support eight 3d artists while having time for adding features or doing bug-fixing. They were doing their stuff and it only needed my involvement when new item-types had to be created.

So with that team and VBuilder we were able to theoretically have several hundreds of content-update iterations or content-extension iterations per day! No idea how many they really did but they did plenty. That's a boost, ain't it?

Interested?

So, after 3 years beeing in use I am considering making it commercially available. It's no magic tool but it allows what I have described above. If you have several 3D artists in your team and you want to free programmer resources, this might be something for you! For me the effort to make it available only makes sense, if there are several teams or people willing to buy this. Thus if you think this is something for you, get in touch with me. Write me an eMail with "VBuilder" as subject. Also let me know what you think of what would be a fair price for you and on how many machines you would like to run it on.

Making it even better

I have a couple of interesting ideas of how to bring VBuilder to the next level. Currently it's focus is triggering the import and processing and building VMOs. This has some backdraws when working with the CMOs inside Virtools "Dev" and I think there are some interesting ways to make it even more efficient. If there is commercial interest, I would bring VBuilder to a new level!

To be continued …

Now this gave you a rough idea what VBuilder is about. In the next part I'll go into some details on how exactly it works.

Last week I posted part 1 where I talked about Shaders, Lua and Blendshapes. I'd like to add to the LUA subject, as it seems to be a commonly asked question, that there are no additional tools in the SDK to deal with custom LUA bindings. The docs suggest to use available solutions.

3D Compass

The 3D Compass is a translation handle/gizmo like many of us know it from other DCC (Digital Content Creation) tools like 3ds max or Maya. The good thing is, that it includes handles for translations on planes too (unlike Maya or Unity3D). I really miss them in tools that only provide translations along one axis.

 

If you look at edges of the pane-translation handles, you will notice an additional arc. These can be used for rotations. I like this solutions, a good enhancement! The yellow pivot does not do a 3-axis translation (I never liked those!) but uniform scaling. This also is a good approach. So overall it's a bit like a universal transformation handle. Therefore there is no variation for rotation-only or scale-only modes. Which means – unless I overlooked it – there is no way to use the 3d compass to do a non-uniform scaling.

Another good addition is the ability to clone easily the selection by holding SHIFT while translating. It does a duplication with few dependencies – the 3d entities get cloned with attributes but not the mesh etc. Good for building content/levels.

The 3D Compass works with the available coordinate systems: local, global, view, parent. (About the ref. guide I am not sure). Something that didn't work yet was the angle snapping when using the rotation handles, but maybe that was fixed for the final release. If not, use the old method. Something that wasn't yet possible too, is the ability to show/hide the 3D compass as in some situations it might be disturbing.

Enhanced Content Protection

VSL code and Shader code can now be encrypted and therefore shared without knowledge transfer. This allows to create a more solid commercial environment for 3rd party component developers and freelancers/consultants.

Protecting scripts

The password can be specified in the variable manager. Having the correct password allows to decrypt (unprotect) the content. I don't know how solid this is but at least it's one more level of obfuscation and should be good enough for most cases.

For some unknown reason are LUA scripts excluded from this protection scheme. Another problem is, that you can't select multiple items in the editors and un-/protect them all in one go. While the Shader Editor handles at least the first selected item, the VSL Editor just doesn't react. So going one-by-one is not very effective, when dealing with more complex content.

New Building Blocks

XML: When Dassault Systemes added XML BuildingBlocks to Virtools 4.0 but restricted them for deployments for VR/XE/Office players I thought "Oh my!". I mean, if you want to create some interesting online content XML as document-format standard is just omni present. For example WebServices. Looks like Dassault Systemes finally became aware of this and now the XML BBs can be used for Webplayer projects too. Personally, I've only tried to use them once under 4.0. They didn't seem to be very stable  (which might have changed in meanwhile) and I ended by dropping them in favor of a custom VSL solution. Anyways, it's still a good addition that should have been done much earlier.

Thus from now on, they can be found under Narratives/XML Parser. The good thing is, that it can be used via VSL too. Moreover there is a visual XML Debugger window. The documentation could be better, I think, especially as the usage is not always totally clear – for example "XML Load Document" may fail but without any hints why.

XML building blocks

Content Processing: I couple of new, useful BBs for examining and processing content: Merge Materials and Merge Textures (to find duplicated/identicals inside a group), Hierarchy Parser Upwards, Dependency Parser , Is Child Of 2D/3D (checks the entire hierarchy upwards)

Camera Movement: Pick And Pan BB, Pick And Rotate, Camera Zoom Extend

The basic idea of BBs for common operations, especially for configurators and the like, is nice. The execution of that idea is "suboptimal". First of all in the beta version the Pick BBs ignore a 2D hit. This means that while a user interacts with the GUI, he might modify 3D content that is hidden behind the GUI without intention. A developer would thus manually check for a 2D hit and deactivate the behaviour, making the "out-of-the-box" idea less effective (less fast 😉 ). The pick and rotate BB rotates a picked object along all 3 axis. No idea where this is useful, because reorienting an object towards a desired orientation is very hard to achieve this way, I think. It would be more universal (= fits to more use-cases) if it has options to constrain it to one axis.

Is Key Down BB:  LOL, finally! No more to say about that 😀 I guess everybody had their own custom solution for that (i.e. via VSL )

Set World size BB: resizing a 3D entity by giving the desired world size.

Spherical/Cartesian coordinates converters: the spherical coordinate consists of a vertical Angle, a horizontal Angle and a distance value. It sounds interesting and I wonder for what user-scenario these have been designed … GeoInformation systems? It also sounds like that we could simulate the content in 2D and display it on a spherical surface i.e. pathfinding and move-to, using the built-in solutions which might be difficult otherwise. Here two images from the Virtools documentation showing the coordinate systems:

2D Texel = Screen Pixel BB: this BB adjusts UV coords of 2D entites to fit texel/screen pixel ratio, a bit like screen mapping. The docs says "This is particularly useful to ensure sharp 2D GUI display by avoiding resizing artefacts".

To be continued …

Ok, I think we covered the major new features so far. In the 3rd part there will be some more and a final conclusion. 

Shaders

If you have been writing PostFX shaders in Virtools, you may have waited long for this, but here it is: overriding techniques for rendering into RenderTargets!!! You can has Tron, yay! Here's a simple example for a masked glow PostFX:

Masked Glow

Overriding techniques allows you to specify a technique name that shall be used when rendering the scene using the Render Scene in RT View BuildingBlock. Previously one was only able to use one shader/material for all objects. That's ok for some tasks but if you need need per-material masking, there was no easy way for doing so. I first expressed the need for a better way in 2005 – that's 4 years ago! In 2006 I tried manual techniques switching and hit another barrier. Now, finally, it's a smooth process. Better late than never!

Some help with hiding or unhiding elements of the scene (i.e. something like RenderLayers) has unfortunately not yet been added. Also some other aspects like include management still needs improvements.

There are a few new shader semantics available:

AlphaTestEnable, AlphaBlendEnable, AlphaRef and for OpenGL only: SingleSided, DoubleSided

If you check the SDK, you will  find traces of WIP (work in progress) for shader-based shadow maps: a new shader semantic, a new BB, a new Rendstate and maybe a future built-in shadow-shader. You will also notice that support for hardware shadow texture formats has been added.

LUA

LUA has been added as scripting language. LUA is a widely used scripting language in the games industry. In contrast to VSL (Vitools Scripting language) LUA is not strongly typed and not JITed (Just-In-Time compiled). It's therefore to be considered to be slower than VSL. So why add it? VSL is very focused on implementing new BuidlingBlocks via scripting. It's not very strong with custom data types and working in a global scope is very limited.

Moreover not everybody likes to use the concept of Schematic Programming. By using the SDK it's possible to bypass it, but of course it makes development slower again. Using LUA one is now able to script a game without using the schematic a lot. This is possible because all LUA scripts share the same context. ( A bit of schematic is still a required though).

As LUA is known by a wider audience, new (script) developers can pick up 3DVIA Virtools much faster without worrying a lot  about the schematic. A good example for this are the 2 last adventure games by City Interactive. Moreover there is a new example game (a BoulderDash clone) entirely writtin in LUA. I think it's also a good starting point for Virtools users that still need to learn LUA!

Lua in Virtools

As LUA is a dynamic language there is no help from a compiler but a button for checking the syntax is available. Setting breakpoints and stepping through the code is possible! There is a small input field in the right side of the toolbar for calling LUA functions directly but i think it's a bit small. From my Maxscripting experience a big input console that allows to prototype interactively is a big plus and hopefully it will come in a later release.

You can't yet use LUA for action scripts and therefore you don't have direct access to the selection. But using the run-button LUA scripts can be executed at any time in authoring mode too.

Personally I am not much attracted by the LUA syntax. Further more I saw some articles on how to implement OO in LUA and they scare me off 😉 Currently I think using C# as embedded scripting language is my favorite! AngelScript recently got simple inheritance and interfaces – it's also a strongly typed language with JITC – might be an interesting alternative to LUA and Co.

The LUA editor control is the same from the shader editor and VSL editor and thereore suffers from the same problems i.e. long scripts slow the entire editor down (due it's slow syntax highlighting I guess).

Blend Shape Support

Blend Shape Support basically means morph targets mixable with skinning (bone-based deformation). So you can have facial animation or body deformations/customizations using morph targets ( blend shapes in Maya) at the same time with your standard bone-based character animations.

Probably most people will think "finally facial animations!" … yeah, cool … but I think you can do much more with it…

😎

What about muscle deformation? Here is a simple example of what I mean: an arm deformed by bones can be morphed simultaneously to bulge it for the muscles effects …

Above you see two arm entities, both are referring to the same mesh, but one is using the morph weights differently. The great thing is: it's still only ONE mesh. You don't need to copy the mesh for each entity! Thus it even works independently for GPU skinned characters – only one mesh and full individual weight control .. I tested it and it seems to work!

Currently when exporting from 3ds max, you need to do a "export selected" that excludes the morph targets otherwise they will be added as additional body parts. Hopefully in future those will be detected and skipped automatically.

I think this is probably one of the new feature that really gives a solid advantage over other 3d authoring solutions! Flexible and simple workflow, nice!

To be continued …

There are more new features but it's getting late (bedtime!). Please note that this is beta experience and some things might be different with the final release. I hope it gives some more insights about 3DVIA Virtools 5's new features as benefits are not always clear via a plain listing of "what's new?" items. Feel free to add informations or to ask questions. Till next part, cheers!

Recently I was asked how to use global variables inside global functions using VSL, one of 3DVIA Virtools built-in scripting languages.

In VSL you can share a variable across diferent "Run VSL" BuildingBlocks by using the keyword "shared". For example

Besides using VSL in BBs or actionscripts, you can also create "global" VSL scripts. These are automatically included. The things is that you can not use the shared keyword inside global scripts. So if you want to modify shared variables inside global functions, one workaround is to use parameter-objects.

Basically it's a struct containing variables you like to share. Either one struct for all or split your variables by context into different structs. This user defined structure makes passing them to functions much easier. Also less code changes are required when adding more data.

Above you see the definition and how it's used as input parameter of a global function. This works because it's passed by reference and not as value (/copy). This way you can modify shared variables even inside global functions. Here is how it could look like inside a VSL BB that calls the global function:

I hope this will help a few more people. If it's not clear, let me know.

CGGenie did an interesting survey called "Upgrades09". After having published the results (see previous link), they also added a very interesting article about how to interpret the results!

It's title: "CG Survey: a question of cost and satisfaction".

To resume the article and the survey, one could say: professional users are less satisfied with their tools than hobbiest or casual users.

Here some extracts:

3ds Max's users have invested a large amount of money, are likely to be professionally pressured in their usage and timescales and also are likely to be pushing the software to its limits every day.

[…] in more challenging ways and those little niggles might become major blockers, those quirky crashes become fundamental cash burners – even though the actual reality of the event, the flaw or the software limitation wouldn't have changed, the user requirements would have.

This helps to understand why a diversified, professional user/customer base might seem always as unhappy grumbler. They use the tools under tight time-frames and budgets. Working with their tool everyday they see what works efficiently and what not. Moreover, they want to push it to it's edges. They want to be fast(er). And if the tool improves, they move fast towards the new 'edge' … pushing it some more! And moreover not necessarily everybody is pushing it into the same direction …

Obviously it's "a far greater challenge" to satisfy those people. 

Last august I finally took some time to try an idea I had or a while (years?). Would it be possible to mis-use 3DVIA Virtools Point-Cloud system for rendering foliage objects (grass, flowers, plants etc). Basically, Point-Clouds are used to visualize 3d scans or data. It's a feature that was slowly introduced with 3.0 Service Packs and finalized in 3.5.

It comes with a feature that is called "Point Cloud Selection" which allows to render separately a subset of the cloud. Interestingly you can also render meshes instead of sprites. This, plus being able to use shaders with point clouds gave me the idea that it might be useful for outdoor foliage rendering.

I'll make it short: it doesn't work the way I hoped.

If you try to render a selection with meshes using a shader, the whole system slows down intensively. I think it's a combination that was not considered or intended by the developers. So it's only compatible with the fixed function GPU pipeline. Besides that I am not sure what it takes to calculate the vertex positions inside the shader correctly, so a mesh is rendered at each point position.

In addition to that, the bigger the cloud is, the slower is the selection process itself. Thus you cannot have one point cloud for your entire outdoor scene – you would have to manage multiple clouds via a grid, nullifying somehow the benefits of this out-of-the-box selection-system.

Here is a picture where on the left you see the whole point cloud and on the right you see a circular selection around the character:

Point cloud grass

Of course, there are other methods to do the same. Justed wanted to see if it goes also like this. :)

For a couple of years now, I am "misusing" Characters for a lot more than "just characters". For those that don't use Virtools, a Character Entity is a 3D entity that has "bodyparts" entities. A bodypart can be a bone, a dummy or a an entity with a (skinned) mesh. You can also add all kind of non bodyparts to the hierarchy.

The interesting aspect is that the whole combination can be treated as one. In the 3D layout window, you usually pick the character entity and thus you translate the complete set. Moreover there is a character parameter type, thus you can search, store, hide etc the whole complex as one.

Office chairs, where you want to be able to change the height or rotate the seat, are a simple example.

Last year I had a combo of particles systems that I wanted to treat as one unit. Easier as asset and easier to create, duplicates etc. Thus I wanted them to be part of a Character.

(Edit: the following passage is not entirely correct, see notice in bold that follows)

The first thing you may notice is you can't easily create any characters or bodyparts inside the Virtools authoring system. Even via the scripting language (VSL) one is not able to create such objects.

Missing VSL bindings is still a problem in Virtools (although things got better). We have a custom DLL where we do our own bindings in these cases, but I wish I would not have to waste my time on this. After I was able to create Characters and bodyparts via VSL I saw that the Level Manager didn't update it's character content. I don't know yet what kind of notification it requires but saving and reloading the file (or asset) fixes the problem.

( EDIT:  as pointed out by Julien in the comments, you actually can create characters and bodyparts using bc.CreateObject – just like in the SDK. There's are only no shortcut bindings like bc.CreateEntity3D )

On my first iteration I simply added the particles system dummies to the character hierarchy, but they also were listed separately when loading them at runtime. Unfortunately you can't attach the particle system components (building blocks) onto bodyparts but only onto so called "3D frames" which are the 3d dummies (/gizmos) of Virtools.

"Hey wait!" came into my mind – 3d dummies in Virtools are 3d entities with an additional flag and bodyparts are special 3d entities. So I did the hack: added the flag to my bodyparts and suddenly they accepted the particle system buildings blocks!

I wrote me a little action script that creates a "particle character" from a set of distinct particles systems. I have no idea how solid this is, but so far no complains from the customer! Hehehe …

😉

It makes it much easier to move, load, store particle-system combos (only textures need to be selected manually in some cases).

Little bugfixing update:

  • fixed: crashes with "Animation Slider" dialog under Virtools 4.1
  • fixed: Canceling the "Add Group" dialog for the "Fast Cam Switcher" crashed Virtools

Thanks for Patrick (aka Hightree) and Antoine (aka Feranti) for reporting them!

I replaced the old file on the server with the new one. The readme and the directory should say "beta 2". Otherwise try a refreshing click on the download-link ( shift+click (?) in firefox ).

Dom's Interface Extensions for Virtools

Let me know if you have any troubbles with the download or the plugin itself. Btw. did you know that you can use the arrow keys in the "Fast Cam Switcher" to move up and down in the cam-list so you can switch between the listed cams really fast?