Hello,
I'm using the Samsung Gear VR Framework to develop a VR 360 video playback app. The video's I will use all have surround sound that need to be lipsync to the video.
It would be best to add to audio to the mp4 video file, but how would you then get that audio so you can rotate it using headtracking? Android mediaplayer will play the audio, but it will not rotate the field when the user rotates his head. The Samsung GVR framework relies on GVRAudioEngine (cardboard / google vr api) classes to playback audio with use of the headtracking (binaural). However, that only works for separate audio files, not linked to video. I'm in the dark on how to sync such audio to a video. The API doens't expose any functions to get/set playback position to allow me to make my own sync engine (I'm even not sure that this would work)
Any pointers are welcome!
Thanks,
Vincent
I'm using the Samsung Gear VR Framework to develop a VR 360 video playback app. The video's I will use all have surround sound that need to be lipsync to the video.
It would be best to add to audio to the mp4 video file, but how would you then get that audio so you can rotate it using headtracking? Android mediaplayer will play the audio, but it will not rotate the field when the user rotates his head. The Samsung GVR framework relies on GVRAudioEngine (cardboard / google vr api) classes to playback audio with use of the headtracking (binaural). However, that only works for separate audio files, not linked to video. I'm in the dark on how to sync such audio to a video. The API doens't expose any functions to get/set playback position to allow me to make my own sync engine (I'm even not sure that this would work)
Any pointers are welcome!
Thanks,
Vincent
No comments:
Post a Comment