Capture rate issue and video rendering of silent audio frames

Capture rate issue and video rendering of silent audio frames

I'm new to video processing and I'm having a hard time with it. In my
application, an animation that lasts 8 seconds (more precisely, the video
contains 262 frames) should be mixed frame by frame (via OpenCV) with a
video recorded by the user (code snippets below).
The animation runs at 30 frames per second (the FrameGrabber for the
animation has a frame rate of 29.97002997002997 fps). Both the
MediaRecorder (that records the user video) and the FFmpegFrameRecorder
(that composes the animation with the user video) are set to run at 30fps
(MediaRecorder.setCaptureRate(30), FFmpegFrameRecorder.setFrameRate(30)).
As investigated by a call to
Camera.getParameters().getSupportedPreviewFpsRange(), the front camera of
my Samsung Galaxy S3 supports a range of 5-31 fps.
After the user video is ready, I check the frame rate of the FrameGrabber
for that recording (FrameGrabber.getFrameRate()), which returns 30. But
the MediaRecorder seems to work at around 15 fps: at least twice as much
time has to be recorded so as to keep the audio from the animation running
as expected, without being silenced at the end of the resulting video.
Because my MediaRecorder setting is not working and the Camera does not
support 60 fps, a workaround was inserted in the recording phase: now the
user has to record at least 16 seconds to produce around the same amount
of frames contained in the animation file. That works, but I feel there is
something wrong with this. I wonder if Camera resolution has a role in all
this, too (the animation will always be 720x480), and I have printed some
values about picture sizes supported, dimension setting for pictures and
preview pictures, info about preview size. I tried to dynamically change
resolution, but it didn't work out well, few configuration options from
CamcorderProfile, used to set the MediaRecorder (nothing exactly like
720x480, the best was to do:
MediaRecorder.setProfile(CamcorderProfile.get(CamcorderProfile.QUALITY_720P))
).
The big problem is to discover how to include silent audio frames in the
beginning of the user video, because that animation will have to start not
in the beginning of the final video, but a few seconds later. I see
FFmpegFrameRecorder does not record a null Frame, not trivial.
I will appreciate any clue. Thank you!
private boolean prepareMediaRecorder() {
m_camera = getCameraInstance();
m_mediaRecorder = new MediaRecorder();
m_camera.unlock();
m_mediaRecorder.setCamera(m_camera);
m_mediaRecorder.setAudioSource(MediaRecorder.AudioSource.CAMCORDER);
m_mediaRecorder.setVideoSource(MediaRecorder.VideoSource.CAMERA);
// CamcorderProfile.QUALITY_720P = "Quality level corresponding to the
720p (1280 x 720) resolution"
// NOT GOOD: CamcorderProfile.QUALITY_480P,
CamcorderProfile.QUALITY_TIME_LAPSE_480P, CamcorderProfile.QUALITY_LOW
m_mediaRecorder.setProfile(CamcorderProfile.get(CamcorderProfile.QUALITY_720P));
// FROM THE METHOD JAVADOC:
// "Note that the recorder cannot guarantee that frames will be
captured at the given rate due
// to camera/encoder limitations. However it tries to be as close as
possible."
m_mediaRecorder.setCaptureRate(FPS_30);
File outDir = new File(S_APP_FOLDER);
int numVids = outDir.list(new FilenameFilter() {
@Override
public boolean accept(File dir, String filename) {
return filename.startsWith("myVideo");
}
}).length;
m_mediaRecorder.setOutputFile(outDir.getAbsolutePath() + "/" +
"myvideo" + numVids + ".mp4");
m_mediaRecorder.setMaxDuration(MEDIA_RECORDER_MAX_DURATION);
// mediaRecorder.setMaxFileSize(5000000); // Set max file size 5M
m_mediaRecorder.setPreviewDisplay(m_cameraSurfaceView.getHolder().getSurface());
try {
m_mediaRecorder.prepare();
} catch (IllegalStateException e) {
releaseMediaRecorder();
return false;
} catch (IOException e) {
releaseMediaRecorder();
return false;
}
return true;
}
// Create a frame recorder to encode resulting video
private FFmpegFrameRecorder createFrameRecorder(String outFilePath) {
FFmpegFrameRecorder recorder = new FFmpegFrameRecorder(outFilePath,
grabberOrig.getImageWidth(),
grabberOrig.getImageHeight(), grabberAudio.getAudioChannels());
// Set audio params
recorder.setAudioCodec(com.googlecode.javacv.cpp.avcodec.AV_CODEC_ID_AAC);
recorder.setSampleFormat(grabberAudio.getSampleFormat());
recorder.setSampleRate(grabberAudio.getSampleRate());
// Set video params
recorder.setVideoCodec(com.googlecode.javacv.cpp.avcodec.AV_CODEC_ID_MPEG4);
recorder.setPixelFormat(com.googlecode.javacv.cpp.avutil.AV_PIX_FMT_YUV420P);
recorder.setFrameRate(FPS_30);
recorder.setVideoBitrate(716800); // PREVIOUSLY USED: 20000000
//recorder.setVideoQuality(2); // 0 means best
recorder.setFormat("mp4");
return recorder;
}