Tip:
Highlight text to annotate it
X
Making of 'Hatsune Miku sings along with my piano'
Thank you very much for many views, comments and favs
on my previous video 'Hatsune Miku sings along with my piano'.
Here I will make some commentary and explanation about the video.
About AR program
Simple AR program using 5 green dots as AR maker.
Processing Handycam image captured via HDMI.
Using outer 4 dots to recognize marker position,
and inner 1 dot to recognize orientation.
Recent AR programs often use feature points in a image,
but I have adopted this simple algorithm
because the response time is important in this program.
As a result, high speed marker recognition was achieved.
This motion was made using MikuMikuDance.
Create motion like this in advance,
pause after each actions,
and proceed triggered by piano key patterns.
This scene is a 2D animation,
however, it was also made by MikuMikuDance.
Everything shown in this scene is MMD models.
Draw a texture by Inkscape,
and create polygon model with just one rectangle.
Making 2D animation by moving them on X-Y plane only.
About piano play synchronization
Singing voice is synchronized with piano play
using triggers of particular key stroke patterns.
Triggers can be placed only at where the singing voice disappears
because this program cannot change singing voice tempo.
This program does not analyzing every piano key strokes,
nor controlling singing voice tempo.
Singing voice tempo is constant
and only start timings of phrases are controlled
by triggers placed on where the singing voice disappears.
For example, start of this phrase is
triggered by 7 chords just before.
CG overay by AR is processed in realtime
so if I move camera, CG position will move according to it.
However the camera is fixed on a tripod.
I need some device to move camera.
Order of triggers is fixed
and never changes according to the piano play.
In other words, there is no function to jump to the specified place
by inputting particular key stroke pattern.
Triggers will be activated when every keys in trigger pattern are hit
within preset time range.
On the other hand, triggers will not be disturbed
by superfluous key stroke.
The song will stop when I miss trigger,
but never affected by misses in non-trigger part.
About my piano play,
I have practiced but this mess.
Not only with my piano play, I'm not satisfied with many points
such as vocaloid tuning or CG motions.
But I posted this video because I thought nothing will begin unless I publish it.
I will devote myself in the future.
The trigger for this action is a chord for squatting down,
and glissando for jumping up.
Music: "PIANO*GIRL"
Music: "Mini no hairanai monogatari"
Music: "TRUE TRUE"
3D model: "Lat-style Miku Ver 2.3"
3D model: "Masisi-style Futome Niku Ver 1.0"
3D model display is based on the source code of 'ARToolKit_MMD' software.
A long time has passed since my previous post.
Now I'm developing the advanced version of this program,
but it may take a little more time to publish.
Please be waited for a little longer.
Next developing items
1. Advanced piano play recognition
1. Advanced piano play recognition 2. Changing tempo of singing voice
1. Advanced piano play recognition 2. Changing tempo of singing voice 3.Tempo control according to the piano play
Thank you for watching.