[ [ [ Digital crowds - Yannis Lehuédé

Making crowds of people "artificially" for the cinema is nothing new, and it is easy to guess why we avoid using thousands of extras. In the digital field, the 1998 film Forrest Gump will be remembered first.
And even more so today, when health conditions are becoming very demanding (due to the fight against the Conv-19 pandemic), the question comes up in many aspects. We will make here a small panorama of the techniques allowing to gather infinite crowds, without sanitary distance nor barrier gestures, the whole digitally ... and aseptically !

Picture of duplicated crowd (without temporal offset)

The first method, the simplest, is to duplicate a compact group of people who have already been filmed.
A simple copy and paste, as it were. The pattern representing this group of people can then be pasted as many times, side by side.
When a crowd is duplicated, even with a time shift in the pattern, the problem of its repetition will arise again when the crowd covers a large area.

Picture of duplicated crowd (temporal offset)

The second image (above) illustrates this roughly. We can then try to introduce various random elements into the distribution of patterns to avoid their repetition.
This can become complex for certain software or situations. A different approach will then be required.

Note: When the camera is moving, just make sure that the crowd is hanging on the floor and not slipping like on a conveyor belt. In this way, we will place these patterns of people on shots in a virtual 3D space in which a (virtual) reproduction of the camera reflects the same movements and characteristics as the one that filmed the shot. It’s simple on paper, but in practice it requires preparation; that’s not the point here.

Assuming that we place the crowd on cardboard boxes, in a virtual 3D space to allow for camera movements, then we might as well place each character on an individual cardboard box, and distribute them more randomly than with the previous method. Placed on particles, these cardboard boxes can thus move and/or avoid colliding with certain objects. In short, it offers more crowd control!
In this case, the idea is to film each character individually (or, if necessary, with different clothes or accessories), and if possible: performing several actions per shot. All this on a background that can be made transparent, as below.

This is the same actress filmed in 15 takes with variations of props and costumes, against a green background. Her movements will then be looped in a loop, without the connection between the beginning and the end being too visible.
Now that the "live action" part of the filming has been completed, the characters can now be detached from their green backgrounds and used individually.

With a semi-transparent cardboard character, it is possible to place them with more or less density on a virtual floor, facing the camera. A particle tool is perfect for this use. Each one will have a movement (the one filmed previously) independent of the others. They will be placed on the floor or in the space with general guidelines and above all: coupled with a dose of randomness. Each particle will be assigned a cardboard box with a randomly moving character, again. This system will ensure a much less redundant distribution of characters than the previous method. In addition, we can give a general movement to our crowd, again with variations for each character.

Note: If the crowd is walking in one direction, it will be a matter of each filmed character walking in the same direction in relation to the camera, which requires more careful preparation than before, with some restrictions. And the most important thing: changes of trajectory are forbidden !.

We can also place collision objects in contact with which the particles (and therefore the characters) will not be able to enter, or will change their behaviour. This is the case in the example below. Simple cubes have been reproduced instead of planters and telephone boxes in the same places as in the filmed shot, where no particles will be created.

Once the floor "populated" with particles of characters, they can be viewed through the camera. To simplify the system, each board will always face the camera. As the characters have been filmed on an erasable background, these boxes are semi-transparent. The camera will see layers of people in front of each other. And as the particles are distributed in space, certain (sanitary!) distances between the characters are respected and they do not pass through each other (as long as the system is not too dense). In this way, waves of the world can be created as far as the eye can see.

In the picture above, we covered the forecourt with people, avoiding the big obstacles. The character particles have fulfilled their function. In a case like this one, we still have to add some masks because there are panels that are supposed to be in front of characters, tree branches, etc.
To do this, several techniques are available in compositing, depending on the shape of the objects that will have to mask the crowd. They include a combination of masks, rotoscoping, movement tracking, keying and deep compositing. (or a variant) to allow some characters to be in front of the mask, others behind, depending on their position in the virtual space.

Last but not least, a final step is to match the brightness and colours between the crowd and the set, as below.

We will also take care to apply the clean grain of the filmed image to our virtual crowd, recreate the eventual blurring of depth of field, as well as to reproduce the same deformations as those of the lens used during the shooting, which is not a problem since our crowd and the camera are in 3D.

For a comparison with previous methods, here is this one applied in the stadium example.

Here too we see a limit: the lack of variety of characters (15 unique characters here) is felt. To remedy this by staying on the same system, it is possible to create slight variations in the colour of clothing and skin in post-production. It is also possible to introduce variations in the speed of the characters’ movements. All of these mixed combinations multiply the variety of "unique" characters in the crowd.
Another element we have at our disposal is to make global colour corrections to each individual in the crowd, in a random way (we will randomly attribute a nuance to each one, which will then guide the intensity of a global correction on the crowd). By combining variants and effects, more and more variety of colour shades can be obtained.

Note: the movements of the individuals making up the crowd remain limited to the cycles of filmed movements of the actors, with small variations. And sometimes redundancies are perceptible where the crowd is not very dense (depending on the situation). There is limited control over the movements of the crowd. And you can’t direct their behaviour, such as simply making them take a turn.

To control the general movements of the crowd while staying in a film setting, you have to rely on a quasi-military choreography. The example of battalions moving in a coordinated manner without a "central" pilot hangs on an even more titanic dimension if each individual has to respect a course without reference to the other members of the group!
There remains the possibility of having a small troop of actors move in different places alternately, and then place them all together on the same terrain. This amounts to falling back on the copy and paste solution, with less latitude than for a static crowd.

The solution to this problem is to use totally virtual characters, in 3D, which is called crowd simulation.
We have seen it in big-budget films (and rare series), and with good reason: it is a more delicate logistics than the previous solutions and a matter for specialists.
This solution naturally offers crowd control that is beyond imagination. In short: each individual has a kind of intelligence and reacts to situations that come up against him or her according to defined rules.
These rules can go as far as : "charge at will and fight to the death" if you lead an army of small soldiers armed with swords and shields. Each individual will race towards the enemy’s ranks, avoiding hitting his "compatriots", and then will search for an opponent within his field of vision on his own. A mock duel battle will follow. Every little virtual soldier risks being mowed down from behind at any moment, as it can be defined that they only trigger reactions according to their field of vision. There is little chance of getting a repetitive crowd with this multitude of truly independent individuals!

All this is governed by libraries of pre-recorded movements which are linked together according to the "stimulations" of the individuals. Each of these movement cycles must first be recorded or animated and the characters modelled individually, and a texture applied to each one. Note that it is also possible to set up variations in the combinations of props and costumes on the virtual puppets.