vfig on 26/12/2023 at 07:02
Quote Posted by PinkDot
However they need to share the palette, if they're .gifs. I hope this works with other formats too - will try next.
i used png textures. any texture format that works for meshes will work here.
Quote Posted by PinkDot
Oh, and the lipsync was actually terrible. No correlation between the words whatsoever. As you pointed out, the job done by face_process isn't great and could be improved using a different solution.
EDIT 2: I actually take back my words on the quality of the lipsync. While it may not be ideal, the reason why it was off sync for me previously was that my speakers were connected via bluetooth.... :rolleyes: Once I switched to normal ones, it felt much better! Yay!
that is interesting! at 16 frames per second, the animation is fairly coarse, and can be off by up to ~ +/- 32ms; i guess that plus the bluetooth latency was enough to disrupt perceived sync, which usually happens at ~80ms. that suggests your speakers over bluetooth had a latency of at least 50ms, which (to me) sounds surprisingly large. though a quick google suggests audio latency via bluetooth is often in the 150ms range, yikes!
Quote Posted by PinkDot
Looks like this isn't even an essential part of the system. Once the facepos.str is generated, the schemas do not need that property anymore.
good to know!
PinkDot on 26/12/2023 at 22:06
Quote:
that is interesting! at 16 frames per second, the animation is fairly coarse, and can be off by up to ~ +/- 32ms; i guess that plus the bluetooth latency was enough to disrupt perceived sync, which usually happens at ~80ms. that suggests your speakers over bluetooth had a latency of at least 50ms, which (to me) sounds surprisingly large. though a quick google suggests audio latency via bluetooth is often in the 150ms range, yikes!
Well, it feels more like 1 second. From my phone to my hi-fi system it's even more like 2 seconds. From what you're saying it seems that this is way out of standard, but I have not gotten time to dig into it yet. I use that for music only anyway...
---
I've made some lipsync test for the actual gameplay to see the impact. For this, I added face textures to first 3 characters we encounter in Running Interference. You can see the results here:
[video=youtube_share;2v9CaJK3rqc]https://youtu.be/2v9CaJK3rqc[/video]
There is a glitch - this hardcoded lipsync sequence going on when they're silent. You can see it on the guards, after they finish they're convo about the torch going out. Not sure why is that - I created facepos.str for entire AI_SPEECH (-289) schema archetype. That's 6362 files in total. Notably - the last two are called 'silenc9s' and 'silenc3s' (silence for 9 and 3 seconds respectively - all 0s), but not sure if those are actually used. But anyway, one would think this should cover all the possible spoken lines, but maybe I (or the face_process command) missed something? Or it is a bug indeed, as I recall seeing the woman guard acting as expected on some of the runs, however the archer was always doing something with his lips even when silent.
Hope there is an easy fix for that issue, as currently it's a deal breaker to me. As cartoonish as this effect looks, I think it does add a bit of emotions to the otherwise life-less faces. And hopefully with some simple Squirrel scripts, we could add some other facial expressions and eyes blinking or closing for unconscious bodies.
vfig on 26/12/2023 at 23:37
first off, its really cool to see a demo of this in an actual in-game context! the mouths arent very visible in the video, but thats probably scaling and video compression (i was watching at 1080).
Quote Posted by PinkDot
There is a glitch - this hardcoded lipsync sequence going on when they're silent. You can see it on the guards, after they finish they're convo about the torch going out. Not sure why is that ... Or it is a bug indeed, as I recall seeing the woman guard acting as expected on some of the runs, however the archer was always doing something with his lips even when silent.
you might be able to get more info by set FaceSpew 1 and set FaceStringSpew 1 - those will show some debug info to the monolog. i think you should get a message for the start and end of each line of dialog. hopefully that will help to figure out what's going on. if thats insufficient, i can run it with a debugger attached and maybe see what is happening?
Quote Posted by PinkDot
Hope there is an easy fix for that issue, as currently it's a deal breaker to me. As cartoonish as this effect looks, I think it does add a bit of emotions to the otherwise life-less faces.
yeah. and i think it helps to lean slightly into the cartoonishness with the mouth poses, just like how the guards are very cartoonish in their searching animations and so on.
Quote Posted by PinkDot
And hopefully with some simple Squirrel scripts, we could add some other facial expressions and eyes blinking or closing for unconscious bodies.
eyes closing could be done independently just with Mesh Textures. death/unconsciousness is easy enough to detect. for other facial expressions, it will be a little more tricky to figure out when to change from one expression to another, but not impossible. speculating, but might need to use darkhooks to get notified when links are created/destroyed, e.g. to change to angry face when an AIAttack link is created. though ive never used darkhooks, and dont quite know how/if it interacts with squirrel...
PinkDot on 27/12/2023 at 21:33
Quote Posted by vfig
first off, its really cool to see a demo of this in an actual in-game context! the mouths arent very visible in the video, but thats probably scaling and video compression (i was watching at 1080).
Yes, the video compression blurs the mouth, especially in the shadow. But it's visible well enough in game. Although it is a subtle effect overall, considering how little time we spend looking close at face of AIs in this game.
Quote Posted by vfig
you might be able to get more info by set FaceSpew 1 and set FaceStringSpew 1 - those will show some debug info to the monolog. i think you should get a message for the start and end of each line of dialog. hopefully that will help to figure out what's going on. if thats insufficient, i can run it with a debugger attached and maybe see what is happening?
Sadly, I did not get any extra messages in the monolog after setting those config variables.
However, the jittering mouths appear randomly. One time, both guards have it, another time only one of them do, sometimes both are fine - mouth stay still, when not talking. Although some lines (self-talk) did not produce any lipsync at all. So, it's a fairly random behaviour so far.
It does not occur with Basso - he's fine - mouth stable, after done speaking.
I don't have much knowledge about the schemas. Maybe the Face Motions property should not be applied to all of them after all (that is, it is being used in run-time, not just for facepos.str creation).
Or maybe some of the sound files were missed from the schemas by face_process command? I'm not sure how to track exactly what schema is being used when etc...
However the random nature of this puts me off from further investigations, to be honest. If we were to have a lipsync and facial expression mod, it needs to be a relatively quick job - for what it is. I'd rather have different faces made via a model. This 2d effect can work only with the original, low poly models and low-res textures.
vfig on 27/12/2023 at 23:26
Quote Posted by PinkDot
But it's visible well enough in game. Although it is a subtle effect overall, considering how little time we spend looking close at face of AIs in this game.
yeah, thats why leaning into the cartoonishness and exaggerating the mouth movements is gonna be important. it will look a little silly, but honestly the ais already do (and are!).
Quote Posted by PinkDot
Sadly, I did not get any extra messages in the monolog after setting those config variables.
just dug up my demo files and tried this. and yeah, thats not working for me either; nor are other ConfigSpew() debug messages like for conversation debugging. just poked into the disassembly, and yeah, all those have been compiled out in the newdark dromeds. so this is gonna need quality time with a debugger to figure out exactly what is happening. (the ConfigSpew messages
are still present in the dbgdrom.exe in the leak, but thats olddark and probably still has the only-one-callback hack active. so rather unhelpful!)
Quote Posted by PinkDot
However, the jittering mouths appear randomly. One time, both guards have it, another time only one of them do, sometimes both are fine - mouth stay still, when not talking. Although some lines (self-talk) did not produce any lipsync at all. So, it's a fairly random behaviour so far.
It does not occur with Basso - he's fine - mouth stable, after done speaking.
i did a few runs of my demo mission above, and it doesnt happen while the scripted dialogue is going, but does happen for the other lines the guards say as they fight afterwards:
[video=youtube;efF-dYmNxRw]https://www.youtube.com/watch?v=efF-dYmNxRw[/video]
couple things to note: firstly, this demo was set up to not need both the speech callback and the conversation callback (because this was done for 1.25 before the patch), so if the problem is a missing callback, it is unsurprising that it doesnt show up during the scripted dialogue. secondly, the facepos.str file in this demo
only has entries for the famous dialogue, so all the other guard lines are gonna be using the fallback. thirdly, it is possible the number of speaking ais matters? my rough understanding of the code is that the number of callbacks was per-ai, not global; but i could be mistaken in this.
a few ideas about the endless flapping:
worst case is, the speech callbacks are not firing properly even in 1.26+ with the apparent fix; if this is the cause, then it is unsolvable without exe changes (realistically, le corbeau debugging it and fixing it in a newdark update). also in the "unfixable" spectrum is if it happens as a result of more than one ai speaking at once; reading the code more would probably uphold or rule out this possibility pretty quickly.
a more hopeful possibility is that it is some property of the schemas/facepos.str entries that is the problem: maybe the schemas
do need Face Motions in order for the callback to be set up when starting the speech? maybe theres a limit on the length of the facepos.str entry? maybe theres a case sensitivity problem between the schema/sample name and the facepos.str entry?
or maybe it is a simple as the fact that, because the fallback animation is endless, if it ever gets called upon, then it never stops. i thought the animation stopping ought to happen from the speech ending, but maybe not? the animation is definitely stopped if there is a facepos.str entry being used and it runs to its end, that is clear from the code.
one thing that would be easy to try is patching out the fallback animation so that its all just the 'mouth closed' pose (index 0). that way, even if the face keeps on being animated, it wouldnt be visible to the player. the animation is in bytes 0x625a88 - 0x625aa7 (inclusive) in dromed.exe v1.27, as seen below. finding it in thief2.exe should be trivial too.
Inline Image:
https://i.imgur.com/wCiYuI4.pngthats not really a
shippable patch. it also looks like it would be easy to patch the code so that if there is no sample, instead of using the fallback, it just turns off the talking flag for that ai. that would be more robust.
obviously asking players to hex-edit their thief exes isnt gonna fly. but e.g. for my wip mission i already have a custom osm that applies binary patches at runtime, in memory, so i can hook rendering. so if the patch suggested above works, then using a custom osm could be the way to ship it. its not hard to do, its easy to make it safe, and because its done by an osm, the mod just needs to have a dml that loads that osm, the same way as fm patches use a dml to load nvscript. this could be made to work for v1.26 and v1.27; obviously it would require an update if a future version of newdark is released.
Quote Posted by PinkDot
Or maybe some of the sound files were missed from the schemas by face_process command? I'm not sure how to track exactly what schema is being used when etc...
if you use
ai_sound_watch 99 (e.g. if the ai objid is 99), then you get monolog output for what the ai speaks. it usually only tells you the concept name, not the actual schema it picks, and definitely not the actual sample name it picks. slightly better than nothing, but not too helpful in the current state. again, this is where running with a debugger would help: we just have to identify where the relevant functions are, and then add breakpoints to those addresses that print the relevant arguments.
Quote Posted by PinkDot
However the random nature of this puts me off from further investigations, to be honest. If we were to have a lipsync and facial expression mod, it needs to be a relatively quick job.
being a quick job was never on the cards for this, i dont think. thats the nature of restoring cut features. if you (or someone else) wants to push on with a mod like this, i am very happy to jump in and help fix these technical problems. but the artwork, sound processing (especially
better sound processing than face_process), testing, and managing the whole project are things i dont have the capacity for (or in the case of art, ability for!).
PinkDot on 28/12/2023 at 11:10
Quote:
the facepos.str file in this demo only has entries for the famous dialogue, so all the other guard lines are gonna be using the fallback.
It's possible, that I incorrectly assume, that my facepos.str covers all the lines. It has 6362 entries and a quick look showed a large number of lines starting with sg1, sg2, sg3 etc... But that would mean, that either facepost.str misses some of the sound file references in schemas or there is another mechanism calling for certain sound files, outside of schemas (speech system is an area I'm not too familiar with in Dark Engine).
What I also notice sometimes is that they stay silent on certain lines - not even lip flapping. Not sure what to think of that.
BTW. Do you know if the speech gets activated only when in view or in certain distance to the player?
Quote:
one thing that would be easy to try is patching out the fallback animation so that its all just the 'mouth closed' pose (index 0).
That's a good idea and that works, as expected. And if you know of some way of patching it, then why not. I would just include the patched executables in the mod, LOL! :)
Quote:
if you use ai_sound_watch 99 (e.g. if the ai objid is 99), then you get monolog output for what the ai speaks.
Thanks for that - that's better than nothing, but yeah, not enough to figure out the issue.
This shows the lines spoken by two guards - 533 and 521:
Code:
[AI(533) Watch:Sound] want "Alert one broadcast" concept "atlevelone" tags "Sense Sound,NearbyFriends 6,Mission 1"
[AI(533) Watch:Sound] Not speaking: Current sound has priority
[AI(533) Watch:Sound] Speech system didn't want to speak
[AI(533) Watch:Sound] want "Alert one broadcast" concept "atlevelone" tags "Sense Sound,NearbyFriends 5,Mission 1"
[AI(533) Watch:Sound] Not speaking: Current sound has priority
[AI(533) Watch:Sound] Speech system didn't want to speak
[AI(521) Watch:Sound] want "Alert one broadcast" concept "atlevelone" tags "Sense Sound,NearbyFriends 5,Mission 1"
[AI(521) Watch:Sound] Not speaking: Current sound has priority
[AI(521) Watch:Sound] Speech system didn't want to speak
[AI(521) Watch:Sound] want "Alert one broadcast" concept "atlevelone" tags "Sense Sound,NearbyFriends 6,Mission 1"
[AI(521) Watch:Sound] Not speaking: Current sound has priority
[AI(521) Watch:Sound] Speech system didn't want to speak
[AI(521) Watch:Sound] want "Alert one broadcast" concept "atlevelone" tags "Sense Sound,NearbyFriends 5,Mission 1"
[AI(521) Watch:Sound] Not speaking: Current sound has priority
[AI(521) Watch:Sound] Speech system didn't want to speak
75 thinking in scripts. Last update 11.000000 sec ago.
Setting next thought to 11.000000
[AI(521) Watch:Sound] want "Alert one broadcast" concept "atlevelone" tags "Sense Sound,NearbyFriends 6,Mission 1"
[AI(521) Watch:Sound] Not speaking: Current sound has priority
[AI(521) Watch:Sound] Speech system didn't want to speak
[AI(521) Watch:Sound] want "Alert one broadcast" concept "atlevelone" tags "Sense Sound,NearbyFriends 5,Mission 1"
[AI(521) Watch:Sound] Not speaking: Current sound has priority
[AI(521) Watch:Sound] Speech system didn't want to speak
[AI(521) Watch:Sound] want "Alert one broadcast" concept "atlevelone" tags "Sense Sound,NearbyFriends 6,Mission 1"
[AI(521) Watch:Sound] Not speaking: Current sound has priority
[AI(521) Watch:Sound] Speech system didn't want to speak
[AI(521) Watch:Sound] want "Alert one broadcast" concept "atlevelone" tags "Sense Sound,NearbyFriends 6,Mission 1"
[AI(521) Watch:Sound] Not speaking: Current sound has priority
[AI(521) Watch:Sound] Speech system didn't want to speak
[AI(521) Watch:Sound] want "Alert one broadcast" concept "atlevelone" tags "Sense Sound,NearbyFriends 6,Mission 1"
[AI(521) Watch:Sound] Not speaking: Current sound has priority
[AI(521) Watch:Sound] Speech system didn't want to speak
[AI(521) Watch:Sound] want "Alert down from one" concept "backtozero" tags "NearbyFriends 6,Mission 1"
[AI(521) Watch:Sound] played concept "Alert down from one"
[AI(521) Watch:Sound] want "Alert zero broadcast" concept "atlevelzero" tags "NearbyFriends 6,Mission 1"
[AI(521) Watch:Sound] Not speaking: Current sound has priority
[AI(521) Watch:Sound] Speech system didn't want to speak
533 was saying their lines, but mouth wouldn't move, the other one had mouth flapping.
When the mono log says something like this:
Code:
[AI(521) Watch:Sound] want "Alert three broadcast" concept "atlevelthree" tags "Investigate True,Sense Sound,NearbyFriends 5,Mission 1"
[AI(521) Watch:Sound] played concept "Alert three broadcast"
the mouth would work as expected.
So, maybe there are two different problems or two manifestation of the same problem - random flapping and staying silent?
Quote:
being a quick job was never on the cards for this, i dont think. thats the nature of restoring cut features. if you (or someone else) wants to push on with a mod like this, i am very happy to jump in and help fix these technical problems. but the artwork, sound processing (especially better sound processing than face_process), testing, and managing the whole project are things i dont have the capacity for (or in the case of art, ability for!).
Yeah, well, I don't have capacity for an advanced debugging of binary code like you do. I can just hope that Le Corbeau still reads those forums and will release 1.28 patch one day, responding to your earlier findings. (no need to remind me how little chances there are - I know it ;) )
If the system worked, I could spend some time on getting the artwork done and ask more people to do some testing. Frankly, I think from the artwork point of view, it's not a difficult job - once we settle on the right shapes of the mouths for each index. The mouths are not very detailed and I don't think this mod would make sense with higher res textures. One reason is how wasteful in terms of texture memory this approach is (especially, if we wanted to include some facial expressions). And secondly, making it higher res, would only expose the flatness of the effect even more. So, creating artwork is relatively easy and quick task, with small adjustments for each set of textures only.
Speaking of the mouth shapes / phonemes:
This enum:
Code:
enum eMouthPos
{
kMouthClosed,
kMouthSmallOh,
kMouthSmallEe,
kMouthBigOh,
kMouthBigEe,
kMouthShout,
kMouthPosCount
};
says one thing but the hex codes from the Dromed.exe indicates something different: AEIOU. Not sure the meaning of the symbol in the binary file, but it may simply reveal the intention. Determining the most adequate shapes for the facepos output would be probably the most important thing from the visual point of view.
For the better results of facepos - I know you mentioned that another application could be used to calculate those. But at the same time, I think that relying on 5 shapes is very limiting. Not sure if any serious application would limit itself to 5 shapes only. Animators typically would use a dozen or so, so even if we remap the "better" output to 5 phonemes only, the quality may not really improve. What could be improved would be increasing number of phonemes supported. Even adding extra 2 or 3 could make for a good compromise. Anything more than that and we step into ridiculous amount of textures that the mod would require.
EDIT: do you think it's possible to patch the binaries to increase the limit? I would imagine wherever the
kMouthPosCount value is referenced, it needs to be replaced with some higher value - would that be enough?
PinkDot on 28/12/2023 at 11:48
I just uploaded 2 videos (combined as 1) recorded yesterday, that demonstrate the issues:
[video=youtube_share;XU98LC9IO9c]https://youtu.be/XU98LC9IO9c[/video]
First 2 sessions are working mostly as expected, although her line at 00:47 ("Not such a bad job...") does not have any mouth movement - not even the flapping.
Second video starts around 1:25 and has a number of game mode sessions - most of them have mouths flapping, but occasionally one or the other character have them OK, at least for some time. In the last session, the sword guard starts acting properly, after I knock out the archer (not sure, if relevant, but could be related to the number of AI speaking).
EDIT:
The "Not such a bad job..." line is file sg6a0mu3.wav and is included in the facepos.str:
Code:
sg6a0mu3:"2420111100002221101100122100000001100000011022111111221000000000000022001100222200000000220000022222442222100000000002022022221100000001110010121222244222221001011000000000011011122222122212100000022122100000000000000000000000000000420000000001122011102211111222002211000"
vfig on 29/12/2023 at 02:08
Quote Posted by PinkDot
It's possible, that I incorrectly assume, that my facepos.str covers all the lines. It has 6362 entries and a quick look showed a large number of lines starting with sg1, sg2, sg3 etc... But that would mean, that either facepost.str misses some of the sound file references in schemas or there is another mechanism calling for certain sound files, outside of schemas (speech system is an area I'm not too familiar with in Dark Engine).
facepos.str contains one line per sample (i.e. per wav file). the face_process command should be finding all the samples from all the schemas that have Face Motions on. from just reading the code alone, i dont think this is the problem. my t2 snd.crf has 6579 wavs; with some maybe unused and i guess some not using Face Motions in your setup, then 6362 entries sounds reasonable. for the moment i am going to assume that missing entries from facepos.str is not the problem.
Quote Posted by PinkDot
BTW. Do you know if the speech gets activated only when in view or in certain distance to the player?
there is a distance cutoff: "if a guard mutters to himself, but no player is close enough to possibly hear it, does he make a sound?" - the game says no, and doesnt play speech. this cutoff is a straight-line distance of 100 units; ai_sound_watch on an ai will print an appropriate message if the speech was cancelled due to distance. however, there is no "in view" cutoff, because the player needs to be able to hear a guard in the next room muttering to themself.
Quote Posted by PinkDot
This shows the lines spoken by two guards - 533 and 521:
Code:
[AI(533) Watch:Sound] want "Alert one broadcast" concept "atlevelone" tags "Sense Sound,NearbyFriends 6,Mission 1"
[AI(533) Watch:Sound] Not speaking: Current sound has priority
[AI(533) Watch:Sound] Speech system didn't want to speak
...
533 was saying their lines, but mouth wouldn't move, the other one had mouth flapping.
When the mono log says something like this:
Code:
[AI(521) Watch:Sound] want "Alert three broadcast" concept "atlevelthree" tags "Investigate True,Sense Sound,NearbyFriends 5,Mission 1"
[AI(521) Watch:Sound] played concept "Alert three broadcast"
the mouth would work as expected.
So, maybe there are two different problems or two manifestation of the same problem - random flapping and staying silent?
those could both be results from the face state getting out of sync from the actual speech, e.g. due to the callbacks not working properly.
i think these are really the crucial two questions: are the speech callbacks being installed/uninstalled correctly for ais with the Face State property; and are the callbacks firing when they should, especially when a Conversation is involved (Conversations install their own speech callbacks so they know when to move from a "Play Sound/Motion" step to the next step).
answering those questions is going to need some quality time with a debugger. if the callbacks are all working correctly, then the bug is probably just in the Face State management, and should be easier to pin down.
Quote Posted by PinkDot
If the system worked, I could spend some time on getting the artwork done and ask more people to do some testing.
thats fair. if the system just doesnt seem reliable enough, why put the time into it? so i agree: figuring out what this bug is and what we can do about it is the next step that needs to happen...
i am interested to do this debugging and chase down this bug, but not yet... i am trying to keep track of too many tasks right now, and another one that needs a lot of focus and attention like this is too much just at the moment.
Quote Posted by PinkDot
but the hex codes from the Dromed.exe indicates something different: AEIOU. Not sure the meaning of the symbol in the binary file, but it may simply reveal the intention.
unrelated. that part of the exe is just storing all the static data used by the code there. the "AEIOUaeiou" string there is used by a helper function for debug messages that checks if the first letter of an object's archetype's name is in that string (i.e. a vowel) in order to decide whether to print it as e.g. "A Chair" or "An Armchair".
Quote Posted by PinkDot
Speaking of the mouth shapes / phonemes: ... Determining the most adequate shapes for the facepos output would be probably the most important thing from the visual point of view.
...
For the better results of facepos - I know you mentioned that another application could be used to calculate those. But at the same time, I think that relying on 5 shapes is very limiting. Not sure if any serious application would limit itself to 5 shapes only. Animators typically would use a dozen or so, so even if we remap the "better" output to 5 phonemes only, the quality may not really improve. What could be improved would be increasing number of phonemes supported. Even adding extra 2 or 3 could make for a good compromise. Anything more than that and we step into ridiculous amount of textures that the mod would require.
EDIT: do you think it's possible to patch the binaries to increase the limit? I would imagine wherever the
kMouthPosCount value is referenced, it needs to be replaced with some higher value - would that be enough?
no. the sFaceNames struct has an array in it of kMouthPosCount size; the whole Face State property is derived from that struct, so this count is essentially embedded inside a whole bunch of memory allocation and member variable address calculations that are baked into the code. patching this is infeasible.
of course, you dont need a
ton of different mouth shapes:
Inline Image:
https://i.imgur.com/N2AzSQK.pngokay, i am including that mostly for fun, but even these reduce down to 6 different face shapes (including closed), if you are willing to forgo detail on characters who, after all, are not having in-camera close-ups:
Inline Image:
https://i.imgur.com/ExIJywH.jpgapart from the kMouthClosed state, which is what is used when talking stops, the ostensible meanings of these different mouth positions in the enum (big oh, small ee, etc.) do not have to be maintained. face_process as it stands does not do any phoneme extraction, but just works on volume alone (hey, it worked for half-life), so already the names have little meaning. but if we wrote a custom python or whatever program to read the .wav files and generate the facepos strings, we can have the other five mouth positions to be whatever we want them to be, as long as they correlate with the output of that program.
in terms of perceived quality of the lipsync, the number of different mouth shapes is much less an issue than the timing of them. for example (and using the 6 colours from the picture above for convenience): for a character saying "too", if the mouth can only goes white, purple, white, this is much less convincing than if it can go white, red, purple, white, with the red position being very short (because the T consonant is very short). the system here runs at a fixed 16 frames per second; that is probably sufficient to do the latter mouth movements instead of the former, just as long as the custom program can do reasonable phoneme extraction.
it is worth pointing out in this hypothetical that we are using all 5 non-closed mouth positions for phoneme variation, instead of volume variation; but since guards in thief both mutter to themselves and yell at the player, having at least one "high volume vowel" mouth shape (like in the existing enum, kMouthShout) is probably a good idea. referencing the coloured wallace groups above, i would maybe sacrifice blue to this cause, and just have F and V phonemes use the same red mouth shape as T,D,K,G,S,N etc. but anyway, my point here is not to make the design decision for the 5 other face positions, but to point out that we can freely make it.
Quote Posted by PinkDot
First 2 sessions are working mostly as expected, although her line at 00:47 ("Not such a bad job...") does not have any mouth movement - not even the flapping. ... The "Not such a bad job..." line is file sg6a0mu3.wav and is included in the facepos.str
ah, this muttering is definitely not all zeros. so yeah, the bug (or bugs) do seem to be affecting starting the animation as well as ending it.
Quote Posted by PinkDot
In the last session, the sword guard starts acting properly, after I knock out the archer (not sure, if relevant, but could be related to the number of AI speaking).
i believe that is a coincidence, as the speech callbacks are per-ai, so the number of speakers should not matter. once i do the actual debugging, i think what is really going on in cases like this should become clear.
vfig on 29/12/2023 at 11:16
Quote Posted by vfig
i am interested to do this debugging and chase down this bug, but not yet... i am trying to keep track of too many tasks right now, and another one that needs a lot of focus and attention like this is too much just at the moment.
ah fuck, my brain had already latched onto this and wouldnt let go. looks like i will be spending a few hours with ghidra and visual studio.
PinkDot on 29/12/2023 at 11:37
Quote Posted by "vifg"
i am interested to do this debugging and chase down this bug, but not yet... i am trying to keep track of too many tasks right now, and another one that needs a lot of focus and attention like this is too much just at the moment.
Sure, we both are involved in other projects and this one is more of a 'nice to have' one.
Quote Posted by "vfig"
no. the sFaceNames struct has an array in it of kMouthPosCount size; the whole Face State property is derived from that struct, so this count is essentially embedded inside a whole bunch of memory allocation and member variable address calculations that are baked into the code. patching this is infeasible.
My understanding is - if you replace the kMouthPosCount in ALL the places it is used - including size of arrays for memory allocation etc. then this could potentially work, no?
Having more mouth indices in our disposal would just open up for more possibilities. But if you're saying these 5 are not even used correctly, then that's a positive thing - a room for some improvement.
Quote Posted by "vifg"
okay, i am including that mostly for fun, but even these reduce down to 6 different face shapes (including closed)
This is now less technical part of the discussion and more creative one - which can be subjective. Speaking of how cartoonish Thief already is - well, there's the original Thief with drunken Benny at the extreme side and there's The Fables of Penitent Thief... I'd rather not go towards the latter. I don't think Thief is cartoonish as such - I would appreciate a bit more subtlety in the lips movement (which would still be cartoony, by the industry standards). But hey - we could have more than one lipsync mod, with different styles, if we have enough room for creative freedom (read - more phonemes available :) ).