Development of organic leg for self balancing characters

[JoeJ]

for me the purpose of this is to have a way to generate what in AI parlance is called the ground truth,
basically, what the AI use to compare results. I am not sold yet on the Reinforming learning what makes the AI learn to move around without guidance. So I will use ground truth in the form of animation to provide style.

This raises some obvious question, assuming i understand your intend.
Why do you need AI at all, if you first need to provide a working example in form of a procedural controllers output?
Wouldn't it be easier and faster to just keep using the procedural controller as is?
If the AI has some advantages, e.g. ability to combine / generalize the functionalities of multiple such controllers, can we achieve the same advantage using a higher level blending mechanism over our procedural controllers?

I don't think that's easy to answer, but i'm pretty sure, finally we'll end up at a combination of both.
It's actually a big difference what's our goals. We work on games, not actual robotics. Our main requirement is not the ability to self balance and do backflips, but to have precise control over timing and to have a predictable outcome.
That's why i personally believe we want procedural controllers first, and only approach ML to address complexities we can not model this way in practice. If we want some game about melee combat, we probably have to model the combat system with manually designed controllers.
Otherwise gameplay itself becomes a form of 'content and assets', similar to graphics shaders. It's not code to make the game - it's content made by artists to have some visual impression.
If our character behavior becomes a matter of NNs, the NN is like an asset, lacking specific control to implement game design in a traditional sense of 'write code so it does precisely what you want.'

But that's just me, inventing an argument to justify and self motivate on my approach, and i'm aware i might be totally wrong.

And that's why...

on the competition, I rather have some cooperation.

... we can't have that yet.
It's too early. We don't know how yet, so cooperation won't help us much because it increases the chance of walking the same path of the same failures.
But competition will help. Only competition allows us to see which techniques give us better results than others and when. Failures of individuals become valuable data for a greater community and turn into success in one way or the other, so no work is lost.

No competition, no progress. That just said to make clear i mean it only positively.

Regarding ML, i just came across an interesting story of a Google engineer which was seemingly fired because he stated in public his belief about true consciousness of the chatbot they worked on. See some example chat: https://cajundiscordian.medium.com/is-lamda-sentient-an-interview-ea64d916d917
I've had a similar discussion with a AI chatbot, but that still failed the Turing test on me. The above discussion does not fail at all. I see a bright future for games, once we can approach truly procedural and interactive stories using such tech.

But that's not really the low hanging fruit i guess. Robotics comes first, and it shall end the current stale state of the industry. Standard is high, all games are pretty good and well done, but nothing pops out anymore. Nothing is truly interesting or exciting, but production costs keep increasing, so a crash seems in front of us, if we can't come up with something new.

Also, with all that character centrism we currently see in modern games, we still lack behind other (true) art forms like books or movies very badly. We can express emotions pretty well from static animation data, but this does not fulfill the dynamic requirements our media imposes. It simply does not really work for what we try to do.

So yeah, 10 years later, i'm still convinced dynamic characters is the next big thing for games. But i still only think so, and do not really know.
I often regret my decision to work on lighting instead, and i can no longer resist the desire to come back to characters, although i really shouldn't but keep focused...

[Julio Jerez]

The benefit of neural net over controller.
Is similar to the benefit of spline modeling over polygonal mesh.

Both spline and neural net fix a hyper curve over a set of data point. Them, from there you can have deviation form those point and predict what points that where not in the data point by interpolation of tge nearest point in the set of known points.

So yes a physical controller would be better, for a given case
But is is very difficult to generalize a physical control using control theory, while neural net you only need data samples and as long as you can train the network, it can interpolate other close samples.

I good case of a neural net application is animation blending.
A net can generate a hyper surface space that is non linear, and can generate much more accurate blend than linear or quadratic or guassian processes or principal component analysis interpolation.

Some big publisher like ubsoft are heavily invested in using neural net for character controller by doing animation blending.

In my case. My idea is to use it for a controller that given a basic pose and the environment, how the pose can be modified to minimize tge error between the desired position and a slight modified environment.

For example.
Say we can get a quadruped walking on a flat terrain.

In this case we have
1 the animation
2 the floor height ( in this case const on each foot)
3 the body should maintain some specified orientation,
4 we can add some other contraint.

That the ground truth. And we train a network to learn that.

Now we play it on a terrain where the floor has some small perturbation,

The job of the net is to produce pose that match the pose from the ground truth and obey the other constraints.

That's a hard problem that is full of state machines and control logic.
But has a generalized way of formating it for a deep neural net.

At lies that how I see the problem, and how I intended to use it.

If I am correct, them I get a trainable controller that can be improved by retraining it with different set of data.
Fir example if the procedural walk is changed to a motion capture set of several animation.

Well anyway, that's my ideas, but I am still far from it.

[JoeJ]

Ha - did i just understand everything you tried to say?
This does not happen so often

Very interesting, i get it, and i agree.
If i predict this correctly, assuming we both succeed, we might get systems of opositting strengths and weaknesses.

That's a hard problem that is full of state machines and control logic.
But has a generalized way of formating it for a deep neural net.

Avoiding state machines (and even state in general) is a primary goal for me too.
A key observation to me was that no state machine is needed for the one legged hopper, and there is no state in memory form the previous steps either.
Instead, all state is given from the current physics, and sensing the current environment.
The hopper does not track states of alternating landing and jumping, it just looks at current velocities etc. to figure out which state it currently has, and reacts by predicting future state of physics using simple models, e.g. inverted pendulum.

So in that sense i'm convinced we can have locomotion controllers respecting dynamic environments, and we can replace state machines with well defined, small optimization systems.

But this only goes so far. My optimism here is bound by the assumption that maintaining balance as the primary objective is enough to give us natural and smart behavior.

Even if this holds for basic locomotion, modeling more complex stuff like interactions of multiple characters, will cause me that same problem of trying to solve it using state machines forming just spaghetti.
ML sounds the proper solution for such 'stuff', but we don't know yet what exactly that stuff is.

Another situation where your work does better than mine is pulling the dog robot down with mouse pick. Your system reacts well automatically already, but i'll have to do some work on that. Likely i'll model this by adding the reaction to external force as secondary objectives to the IK solver, but your system will always react more naturally by respecting a more detailed state of individual bodies physics, i'm sure.

Well anyway, that's my ideas, but I am still far from it.

I guess nobody expects quick results.
When it's done... ;D

Please take a look at my other Newton4 noob question, if you get at it...

[Julio Jerez]

well moving on, I am now making a very basics walk gait.
I am looking at videos like the one below
https://www.youtube.com/watch?v=tLrRlXxM5Yw&t=3s

they show some basic rule how to make quadruped walk gait, and the is that sequence.
it turns out the from walk, crawl and I think few other the walk is the same, but the sequence is
for example give a walk gait, a sequence for crawl, it maximizes the number of limbs in contact with the floor, the reduce to tow sequences
(0.0, 0.5, 0.75, 0.25) or (0.0, 0.5, 0.25, 0.75)
those two manage to keep three out of the four limbs on the ground, but of the two the first one maximizes the area of the contacts.
for a stride walk the sequence changes to (0.0, 0.25, 0.5, 0.75)

which is just two legs in contact while the other two moves forward, this is in fact a managed control falling. as the two legs on the air moves forward, the body will fall to the side, and that is what generates the sway motions.

as you can see these are ethe kind of heuristic that a state doing traditional control logic has resolve but using state machine spaghetti control logic, but a neural network will have no problem learning those parameters. Spatially some special kind of neural net called recurrent network and autoencoders, that has the capacity of learning sequences.

I am now focus on making the crawl walk sequence.
If you sync you will see the dog fixed to the world, so that I can check that the sequence is in fact optimal.

them after I get a good-looking sequence, I will try letting walk.
and that the next challenge, because they it has to deal with the physics for the walk.
by this I mean, if the dog just start to move, the motion breaks the friction, so it has to easy in until it reaches top speed.
that's another learning parameters, that NNs are good at learning.

as you can see are you start mapping the physics, you can see that almost every control starts to look as a knob that you control, and if the physics is reliable and fairly accurate, them the AI should be able to produce a good map of the environment in turn of control knobs.
the is not scenario in which I am planning NN doing the physics like I'd seen some nutjobs out there predicting, including the Google crackpot who thought that, and bunch of numbers became sentient.

there is a lot of misinformation and exaggeration about what neural network can so out there and some people, including the big tech companies are using it for very unscrupulous, unethical and I say some time even immoral goal and purpose.

try sync later so that you can see the state it is at.

[Julio Jerez]

in fact, if you sync

it let it drops of the floor, and it is possible to see glance of how it is going to work.

if you drag the slide slowly you can make take step in the right direction, but since the walk still does no know what foot to lift when moving the leg back, they motion back fight the motion forward,
so I will fix that next.

but so far this is no too bad.

[Julio Jerez]

guys, sorry to keep posting minor updates.

but if you get time try to check it out, I now have the animation channel with the horizontal and vertical track.

it looks so nice, (at least in my vias opinion) that I decide to make a check point.

notice how, like the video say, the crawl walk cycle always has three feet planted on the floor.
that seems like a good sign.

[Julio Jerez]

and the eagle has landed.
the dog is walking.

there are few parameters that can be tweaks. but for now this is a sweep spot.

[JoeJ]

Yay, that's how a proper video game of the 21nd century has to look like.
Goodbye keyframes...

[Julio Jerez]

ok, now I tweaked the animation so that is can walk at the entire range of the animation, but on a flat floor.
so far my assumption about locomotion seem to be sound.

what now have a walk quadruped, that we can use as ground truth, to train a
reinforcement recurrent neural neural net base on un even terrain.

there are still many challenging issues, but my goal is that the model will get the style from the animation rather that form random tries.

I still have to elaborate more on the animations,
It need few more: stance Trot, gallop, and some cues moves.
play it also need to turns.

them after that we start trying Reinforcement learning.
I am working both things simultaneously, so progress is slow.

[Bird]

Yay, that's how a proper video game of the 21nd century has to look like.
Goodbye keyframes...

Hehe, "control the character using natural language commands"
https://www.youtube.com/watch?v=8oIQy6f ... nel=NVIDIA

https://www.youtube.com/watch?v=hmV4v_E ... =JasonPeng

[JoeJ]

Hehe... first they force tensor cores to the masses.
Years later they come up with first ideas on what they could be eventually used for

[Julio Jerez]

The dirty little secret of neural net control. Is that the bandwidth needed is order of magnitude the bandwidth of graphic at the same level.

The evaluation of those convolution and recurrent net, of many level deep, takes several hundreds of million of operation for every evaluation. Also the training of those nets requires so much logistic to get reliably training data, that only big publishers like ubsoft, Activision or dev like unreal and unity can afford.

I am developing a neural net solver because my point is that Nvidia, Intel, Amd, Apple and Google are bamboozling the public making it seen more difficult than it really is.
Neural net are regression machines, given a set of data, they can fit a complex multidimensional hyper surface. Them from there, given a new point the multidimensional curve can tell the user the closest match to that point and use it as the answer.

But these companies are moving everything to the neural net, just because they can spend several millions on logistic to generate the training data.

But just remember, just like Elon Musk said the self driving car will replace everything, but that not happens. neural net are no the answer to all problems. If you do no believe me jut as k the super moron Google engineer who though the machine became Skynet and got himself fired.

Nvidia is spending money because is the only way they can justify people buying hardware to replace the hardware they bought last year. And with crypto currency bubble burning, it is a hard sale.

[Julio Jerez]

Guys, I just tested the neural net for the first time.

And the resuls are impresive.
I am just testing a simple boolean operation, just to make sure it is working.
I did not expect to work the first time but there we have it.

Later this week I will try typical example. Like the hand write character recognition, and see how that works before moving to the reinformen learning part.

I also have to add some metrics code.

[Julio Jerez]

in the simple Boolean operation, that I am using to test it, it is supposed to generate an and and or from two binary input from 0 to 1.
the training input is 1000 randomly generated sample of pairs. After training for 4000 iteration this is how the network converge to a near perfect classifier.

Untitled.png (14.01 KiB) Viewed 11131 times

first the error is large, but then after about 600 passes, is start to converge very quickly.
them as if start ceasing on the lower point, it mades small progress, but that is expected.

now, it think it is reading for more complex test cases.

[Julio Jerez]

I am now trying to train the net to recognize
http://yann.lecun.com/exdb/mnist/

and so far the result are impressive.
this is how the error matrix goes down after about 1100 iteration of back propagation.

Untitled.png (15.82 KiB) Viewed 11103 times

it goes almost steadily down and finds a little bump here and there but that is normal,

I have no test it against the test data, I will that later tonight, but first I have to add some functionality like saving it, and probably the multicore execution to the trainer

the net has 5 layer and 256 neurons. and the training with a single core took about one hour.

so far I am very happy with these preliminary results.