
Recherche avancée
Autres articles (95)
-
Personnaliser en ajoutant son logo, sa bannière ou son image de fond
5 septembre 2013, parCertains thèmes prennent en compte trois éléments de personnalisation : l’ajout d’un logo ; l’ajout d’une bannière l’ajout d’une image de fond ;
-
Les tâches Cron régulières de la ferme
1er décembre 2010, parLa gestion de la ferme passe par l’exécution à intervalle régulier de plusieurs tâches répétitives dites Cron.
Le super Cron (gestion_mutu_super_cron)
Cette tâche, planifiée chaque minute, a pour simple effet d’appeler le Cron de l’ensemble des instances de la mutualisation régulièrement. Couplée avec un Cron système sur le site central de la mutualisation, cela permet de simplement générer des visites régulières sur les différents sites et éviter que les tâches des sites peu visités soient trop (...) -
Personnaliser les catégories
21 juin 2013, parFormulaire de création d’une catégorie
Pour ceux qui connaissent bien SPIP, une catégorie peut être assimilée à une rubrique.
Dans le cas d’un document de type catégorie, les champs proposés par défaut sont : Texte
On peut modifier ce formulaire dans la partie :
Administration > Configuration des masques de formulaire.
Dans le cas d’un document de type média, les champs non affichés par défaut sont : Descriptif rapide
Par ailleurs, c’est dans cette partie configuration qu’on peut indiquer le (...)
Sur d’autres sites (9856)
-
Data Privacy Day 2020
27 janvier 2020, par Matthieu Aubry — Privacy -
Merge multi channel audio buffers into one CMSampleBuffer
26 avril 2020, par DarkwonderI am using FFmpeg to access an RTSP stream in my macOS app.



REACHED GOALS : I have created a tone generator which creates single channel audio and returns a CMSampleBuffer. The tone generator is used to test my audio pipeline when the video's fps and audio sample rates are changed.



GOAL : The goal is to merge multi-channel audio buffers into a single CMSampleBuffer.



Audio data lifecyle :



AVCodecContext* audioContext = self.rtspStreamProvider.audioCodecContext;
 if (!audioContext) { return; }

 // Getting audio settings from FFmpegs audio context (AVCodecContext).
 int samplesPerChannel = audioContext->frame_size;
 int frameNumber = audioContext->frame_number;
 int sampleRate = audioContext->sample_rate;
 int fps = [self.rtspStreamProvider fps];

 int calculatedSampleRate = sampleRate / fps;

 // NSLog(@"\nSamples per channel = %i, frames = %i.\nSample rate = %i, fps = %i.\ncalculatedSampleRate = %i.", samplesPerChannel, frameNumber, sampleRate, fps, calculatedSampleRate);

 // Decoding the audio data from a encoded AVPacket into a AVFrame.
 AVFrame* audioFrame = [self.rtspStreamProvider readDecodedAudioFrame];
 if (!audioFrame) { return; }

 // Extracting my audio buffers from FFmpegs AVFrame.
 uint8_t* leftChannelAudioBufRef = audioFrame->data[0];
 uint8_t* rightChannelAudioBufRef = audioFrame->data[1];

 // Creating the CMSampleBuffer with audio data.
 CMSampleBufferRef leftSampleBuffer = [CMSampleBufferFactory createAudioSampleBufferUsingData:leftChannelAudioBufRef channelCount:1 framesCount:samplesPerChannel sampleRate:sampleRate];
// CMSampleBufferRef rightSampleBuffer = [CMSampleBufferFactory createAudioSampleBufferUsingData:packet->data[1] channelCount:1 framesCount:samplesPerChannel sampleRate:sampleRate];

 if (!leftSampleBuffer) { return; }
 if (!self.audioQueue) { return; }
 if (!self.audioDelegates) { return; }

 // All audio consumers will receive audio samples via delegation. 
 dispatch_sync(self.audioQueue, ^{
 NSHashTable *audioDelegates = self.audioDelegates;
 for (id<audiodataproviderdelegate> audioDelegate in audioDelegates)
 {
 [audioDelegate provider:self didOutputAudioSampleBuffer:leftSampleBuffer];
 // [audioDelegate provider:self didOutputAudioSampleBuffer:rightSampleBuffer];
 }
 });
</audiodataproviderdelegate>



CMSampleBuffer containing audio data creation :



import Foundation
import CoreMedia

@objc class CMSampleBufferFactory: NSObject
{

 @objc static func createAudioSampleBufferUsing(data: UnsafeMutablePointer<uint8> ,
 channelCount: UInt32,
 framesCount: CMItemCount,
 sampleRate: Double) -> CMSampleBuffer? {

 /* Prepare for sample Buffer creation */
 var sampleBuffer: CMSampleBuffer! = nil
 var osStatus: OSStatus = -1
 var audioFormatDescription: CMFormatDescription! = nil

 var absd: AudioStreamBasicDescription! = nil
 let sampleDuration = CMTimeMake(value: 1, timescale: Int32(sampleRate))
 let presentationTimeStamp = CMTimeMake(value: 0, timescale: Int32(sampleRate))

 // NOTE: Change bytesPerFrame if you change the block buffer value types. Currently we are using double.
 let bytesPerFrame: UInt32 = UInt32(MemoryLayout<float32>.size) * channelCount
 let memoryBlockByteLength = framesCount * Int(bytesPerFrame)

// var acl = AudioChannelLayout()
// acl.mChannelLayoutTag = kAudioChannelLayoutTag_Stereo

 /* Sample Buffer Block buffer creation */
 var blockBuffer: CMBlockBuffer?

 osStatus = CMBlockBufferCreateWithMemoryBlock(
 allocator: kCFAllocatorDefault,
 memoryBlock: nil,
 blockLength: memoryBlockByteLength,
 blockAllocator: nil,
 customBlockSource: nil,
 offsetToData: 0,
 dataLength: memoryBlockByteLength,
 flags: 0,
 blockBufferOut: &blockBuffer
 )

 assert(osStatus == kCMBlockBufferNoErr)

 guard let eBlock = blockBuffer else { return nil }

 osStatus = CMBlockBufferFillDataBytes(with: 0, blockBuffer: eBlock, offsetIntoDestination: 0, dataLength: memoryBlockByteLength)
 assert(osStatus == kCMBlockBufferNoErr)

 TVBlockBufferHelper.fillAudioBlockBuffer(blockBuffer,
 audioData: data,
 frames: Int32(framesCount))
 /* Audio description creations */

 absd = AudioStreamBasicDescription(
 mSampleRate: sampleRate,
 mFormatID: kAudioFormatLinearPCM,
 mFormatFlags: kLinearPCMFormatFlagIsPacked | kLinearPCMFormatFlagIsFloat,
 mBytesPerPacket: bytesPerFrame,
 mFramesPerPacket: 1,
 mBytesPerFrame: bytesPerFrame,
 mChannelsPerFrame: channelCount,
 mBitsPerChannel: 32,
 mReserved: 0
 )

 guard absd != nil else {
 print("\nCreating AudioStreamBasicDescription Failed.")
 return nil
 }

 osStatus = CMAudioFormatDescriptionCreate(allocator: kCFAllocatorDefault,
 asbd: &absd,
 layoutSize: 0,
 layout: nil,
// layoutSize: MemoryLayout<audiochannellayout>.size,
// layout: &acl,
 magicCookieSize: 0,
 magicCookie: nil,
 extensions: nil,
 formatDescriptionOut: &audioFormatDescription)

 guard osStatus == noErr else {
 print("\nCreating CMFormatDescription Failed.")
 return nil
 }

 /* Create sample Buffer */
 var timmingInfo = CMSampleTimingInfo(duration: sampleDuration, presentationTimeStamp: presentationTimeStamp, decodeTimeStamp: .invalid)

 osStatus = CMSampleBufferCreate(allocator: kCFAllocatorDefault,
 dataBuffer: eBlock,
 dataReady: true,
 makeDataReadyCallback: nil,
 refcon: nil,
 formatDescription: audioFormatDescription,
 sampleCount: framesCount,
 sampleTimingEntryCount: 1,
 sampleTimingArray: &timmingInfo,
 sampleSizeEntryCount: 0, // Must be 0, 1, or numSamples.
 sampleSizeArray: nil, // Pointer ot Int. Don't know the size. Don't know if its bytes or bits?
 sampleBufferOut: &sampleBuffer)
 return sampleBuffer
 }

}
</audiochannellayout></float32></uint8>



CMSampleBuffer gets filled with raw audio data from FFmpeg's data :



@import Foundation;
@import CoreMedia;

@interface BlockBufferHelper : NSObject

+(void)fillAudioBlockBuffer:(CMBlockBufferRef)blockBuffer
 audioData:(uint8_t *)data
 frames:(int)framesCount;



@end

#import "TVBlockBufferHelper.h"

@implementation BlockBufferHelper

+(void)fillAudioBlockBuffer:(CMBlockBufferRef)blockBuffer
 audioData:(uint8_t *)data
 frames:(int)framesCount
{
 // Possibly dev error.
 if (framesCount == 0) {
 NSAssert(false, @"\nfillAudioBlockBuffer/audioData/frames will not be able to fill an blockBuffer which has no frames.");
 return;
 }

 char *rawBuffer = NULL;

 size_t size = 0;

 OSStatus status = CMBlockBufferGetDataPointer(blockBuffer, 0, &size, NULL, &rawBuffer);
 if(status != noErr)
 {
 return;
 }

 memcpy(rawBuffer, data, framesCount);
}

@end




The
LEARNING Core Audio
book from Chris Adamson/Kevin Avila points me toward a multi channel mixer. 
The multi channel mixer should have 2-n inputs and 1 output. I assume the output could be a buffer or something that could be put into aCMSampleBuffer
for further consumption.


This direction should lead me to
AudioUnits
,AUGraph
and theAudioToolbox
. I don't understand all of these classes and how they work together. I have found some code snippets on SO which could help me but most of them useAudioToolBox
classes and don't useCMSampleBuffers
as much as I need.


Is there another way to merge audio buffers into a new one ?



Is creating a multi channel mixer using AudioToolBox the right direction ?


-
ffmpeg : Combine/merge multiple mp4 videos - not working
9 juin 2016, par dtbakerHere is the command I am using to combine multiple videos :
ffmpeg -i 75_540_38HQ2.mp4 -i 76_70_20.mp4 -i 76_173_80.mp4 -i 81_186_35.mp4 -vcodec copy -acodec copy Mux1.mp4
The resulting
Mux1.mp4
does not contain all videos. Only the first video (75_540_38HQ2.mp4
). The file size of the source and resulting video is below (as you can see, resulting video is slightly larger than first vid) :$ ls -lh
rw-r—r— 1 dbaker dbaker 42M 2011-03-24 11:59 75_540_38HQ2.mp4
rw-r—r— 1 dbaker dbaker 236M 2011-03-24 12:09 76_173_80.mp4
rw-r—r— 1 dbaker dbaker 26M 2011-03-24 12:05 76_70_20.mp4
rw-r—r— 1 dbaker dbaker 54M 2011-03-24 12:15 81_186_35.mp4
rw-r—r— 1 dbaker dbaker 44M 2011-03-24 14:48 Mux1.mp4
Here is the output of the
ffmpeg
command. To me it looks ok, showing the multiple source inputs and the single output.FFmpeg version SVN-r26402, Copyright (c) 2000-2011 the FFmpeg developers built on Mar 21 2011 18:05:32 with gcc 4.4.5 configuration : —enable-gpl —enable-version3 —enable-nonfree —enable-postproc —enable-libfaac —enable-libmp3lame —enable-libopencore-amrnb —enable-libopencore-amrwb —enable-libtheora —enable-libvorbis —enable-libvpx —enable-libx264 —enable-libxvid —enable-x11grab libavutil 50.36. 0 / 50.36. 0 libavcore 0.16. 1 / 0.16. 1 libavcodec 52.108. 0 / 52.108. 0 libavformat 52.93. 0 / 52.93. 0 libavdevice 52. 2. 3 / 52. 2. 3 libavfilter 1.74. 0 / 1.74. 0 libswscale 0.12. 0 / 0.12. 0 libpostproc 51. 2. 0 / 51. 2. 0 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from ’75_540_38HQ2.mp4’ : Metadata : major_brand : isom minor_version : 512 compatible_brands : isomiso2avc1mp41 creation_time : 1970-01-01 00:00:00 encoder : Lavf52.93.0 Duration : 00:00:29.99, start : 0.000000, bitrate : 11517 kb/s Stream #0.0(eng) : Video : h264, yuv420p, 1280x960 [PAR 1:1 DAR 4:3], 11575 kb/s, 29.94 fps, 29.97 tbr, 30k tbn, 59.94 tbc Metadata : creation_time : 1970-01-01 00:00:00 Stream #0.1(eng) : Audio : aac, 48000 Hz, stereo, s16, 127 kb/s Metadata : creation_time : 1970-01-01 00:00:00 Input #1, mov,mp4,m4a,3gp,3g2,mj2, from ’76_70_20.mp4’ : Metadata : major_brand : isom minor_version : 512 compatible_brands : isomiso2avc1mp41 creation_time : 1970-01-01 00:00:00 encoder : Lavf52.93.0 Duration : 00:00:19.98, start : 0.000000, bitrate : 10901 kb/s Stream #1.0(eng) : Video : h264, yuv420p, 1280x960 [PAR 1:1 DAR 4:3], 10804 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc Metadata : creation_time : 1970-01-01 00:00:00 Stream #1.1(eng) : Audio : aac, 48000 Hz, stereo, s16, 128 kb/s Metadata : creation_time : 1970-01-01 00:00:00 Input #2, mov,mp4,m4a,3gp,3g2,mj2, from ’76_173_80.mp4’ : Metadata : major_brand : isom minor_version : 512 compatible_brands : isomiso2avc1mp41 creation_time : 1970-01-01 00:00:00 encoder : Lavf52.93.0 Duration : 00:03:09.99, start : 0.000000, bitrate : 10393 kb/s Stream #2.0(eng) : Video : h264, yuv420p, 1280x960 [PAR 1:1 DAR 4:3], 10321 kb/s, 29.96 fps, 29.97 tbr, 30k tbn, 59.94 tbc Metadata : creation_time : 1970-01-01 00:00:00 Stream #2.1(eng) : Audio : aac, 48000 Hz, stereo, s16, 128 kb/s Metadata : creation_time : 1970-01-01 00:00:00
Seems stream 0 codec frame rate differs from container frame rate : 119.88 (120000/1001) -> 30000.00 (30000/1)
Input #3, mov,mp4,m4a,3gp,3g2,mj2, from ’81_186_35.mp4’ :
Metadata :
major_brand : isom
minor_version : 512
compatible_brands : isomiso2avc1mp41
creation_time : 1970-01-01 00:00:00
encoder : Lavf52.93.0
Duration : 00:00:35.00, start : 0.000000, bitrate : 12700 kb/s
Stream #3.0(eng) : Video : h264, yuv420p, 1280x720 [PAR 1:1 DAR 16:9], 12620 kb/s, 59.91 fps, 30k tbr, 60k tbn, 119.88 tbc
Metadata :
creation_time : 1970-01-01 00:00:00
Stream #3.1(eng) : Audio : aac, 48000 Hz, stereo, s16, 128 kb/s
Metadata :
creation_time : 1970-01-01 00:00:00
Output #0, mp4, to ’Mux1.mp4’ :
Metadata :
major_brand : isom
minor_version : 512
compatible_brands : isomiso2avc1mp41
creation_time : 1970-01-01 00:00:00
encoder : Lavf52.93.0
Stream #0.0(eng) : Video : libx264, yuv420p, 1280x960 [PAR 1:1 DAR 4:3], q=2-31, 11575 kb/s, 30k tbn, 29.97 tbc
Metadata :
creation_time : 1970-01-01 00:00:00
Stream #0.1(eng) : Audio : libfaac, 48000 Hz, stereo, 128 kb/s
Metadata :
creation_time : 1970-01-01 00:00:00
Stream mapping :
Stream #0.0 -> #0.0
Stream #2.1 -> #0.1
Press [q] to stop encoding
frame= 883 fps=632 q=-1.0 Lsize= 44730kB time=29.40 bitrate=12465.1kbits/s
video:41678kB audio:2969kB global headers:0kB muxing overhead 0.184548%Am I doing something blindingly stupid here ?
The source videos came from a video camera, and are small snippets taken with
ffmpeg -i bigfile.mp4 -ss 20 -t 10 -vcodec copy etc..
Thanks heaps !!
Dave
Edit : couldn’t solve it so I just use avidemux GUI tool. It seemed to append the MP4’s just fine.
Must be a problem with MP4’s or just the ones that come off a gopro camera.