Recherche avancée

Sur d’autres sites (3034)

• Parsing The Clue Chronicles

  30 décembre 2018, par Multimedia Mike — Game Hacking

  A long time ago, I procured a 1999 game called Clue Chronicles : Fatal Illusion, based on the classic board game Clue, a.k.a. Cluedo. At the time, I was big into collecting old, unloved PC games so that I could research obscure multimedia formats.

  
  
  

  Surveying the 3 CD-ROMs contained in the box packaging revealed only Smacker (SMK) videos for full motion video which was nothing new to me or the multimedia hacking community at the time. Studying the mix of data formats present on the discs, I found a selection of straightforward formats such as WAV for audio and BMP for still images. I generally find myself more fascinated by how computer games are constructed rather than by playing them, and this mix of files has always triggered a strong “I could implement a new engine for this !” feeling in me, perhaps as part of the ScummVM project which already provides the core infrastructure for reimplementing engines for 2D adventure games.

  Tying all of the assets together is a custom high-level programming language. I have touched on this before in a blog post over a decade ago. The scripts are in a series of files bearing the extension .ini (usually reserved for configuration scripts, but we’ll let that slide). A representative sample of such a script can be found here :

  clue-chronicles-scarlet-1.txt

  What Is This Language ?
  At the time I first analyzed this language, I was still primarily a C/C++-minded programmer, with a decent amount of Perl experience as a high level language, and had just started to explore Python. I assessed this language to be “mildly object oriented with C++-type comments (‘//’) and reliant upon a number of implicit library functions”. Other people saw other properties. When I look at it nowadays, it reminds me a bit more of JavaScript than C++. I think it’s sort of a Rorschach test for programming languages.

  Strangely, I sort of had this fear that I would put a lot of effort into figuring out how to parse out the language only for someone to come along and point out that it’s a well-known yet academic language that already has a great deal of supporting code and libraries available as open source. Google for “spanish dolphins far side comic” for an illustration of the feeling this would leave me with.

  It doesn’t matter in the end. Even if such libraries exist, how easy would they be to integrate into something like ScummVM ? Time to focus on a workable approach to understanding and processing the format.

  Problem Scope
  So I set about to see if I can write a program to parse the language seen in these INI files. Some questions :

  1. How large is the corpus of data that I need to be sure to support ?
  2. What parsing approach should I take ?
  3. What is the exact language format ?
  4. Other hidden challenges ?

  To figure out how large the data corpus is, I counted all of the INI files on all of the discs. There are 138 unique INI files between the 3 discs. However, there are 146 unique INI files after installation. This leads to a hidden challenge described a bit later.

  What parsing approach should I take ? I worried a bit too much that I might not be doing this the “right” way. I’m trying to ignore doubts like this, like how “SQL Shame” blocked me on a task for a little while a few years ago as I concerned myself that I might not be using the purest, most elegant approach to the problem. I know I covered language parsing a lot time ago in university computer science education and there is a lot of academic literature to the matter. But sometimes, you just have to charge in and experiment and prototype and see what falls out. In doing so, I expect to have a better understanding of the problems that need to solved and the right questions to ask, not unlike that time that I wrote a continuous integration system from scratch because I didn’t actually know that “continuous integration” was the keyword I needed.

  Next, what is the exact language format ? I realized that parsing the language isn’t the first and foremost problem here– I need to know exactly what the language is. I need to know what the grammar are keywords are. In essence, I need to reverse engineer the language before I write a proper parser for it. I guess that fits in nicely with the historical aim of this blog (reverse engineering).

  Now, about the hidden challenges– I mentioned that there are 8 more INI files after the game installs itself. Okay, so what’s the big deal ? For some reason, all of the INI files are in plaintext on the CD-ROM but get compressed (apparently, according to file size ratios) when installed to the hard drive. This includes those 8 extra INI files. I thought to look inside the CAB installation archive file on the CD-ROM and the files were there… but all in compressed form. I suspect that one of the files forms the “root” of the program and is the launching point for the game.

  Parsing Approach
  I took a stab at parsing an INI file. My approach was to first perform lexical analysis on the file and create a list of 4 types : symbols, numbers, strings, and language elements ([]{}()=., :). Apparently, this is the kind of thing that Lex/Flex are good at. This prototyping tool is written in Python, but when I port this to ScummVM, it might be useful to call upon the services of Lex/Flex, or another lexical analyzer, for there are many. I have a feeling it will be easier to use better tools when I understand the full structure of the language based on the data available.
  
  The purpose of this tool is to explore all the possibilities of the existing corpus of INI files. To that end, I ran all 138 of the plaintext files through it, collected all of the symbols, and massaged the results, assuming that the symbols that occurred most frequently are probably core language features. These are all the symbols which occur more than 1000 times among all the scripts :

     6248 false
     5734 looping
     4390 scripts
     3877 layer
     3423 sequentialscript
     3408 setactive
     3360 file
     3257 thescreen
     3239 true
     3008 autoplay
     2914 offset
     2599 transparent
     2441 text
     2361 caption
     2276 add
     2205 ge
     2197 smackanimation
     2196 graphicscript
     2196 graphic
     1977 setstate
     1642 state
     1611 skippable
     1576 desc
     1413 delayscript
     1298 script
     1267 seconds
     1019 rect
  

  About That Compression
  I have sorted out at least these few details of the compression :

  bytes 0-3    "COMP" (a pretty strong sign that this is, in fact, compressed data)
  bytes 4-11   unknown
  bytes 12-15  size of uncompressed data
  bytes 16-19  size of compressed data (filesize - 20)
  bytes 20-    compressed payload
  

  The compression ratios are on the same order of gzip. I was hoping that it was stock zlib data. However, I have been unable to prove this. I wrote a Python script that scrubbed through the first 100 bytes of payload data and tried to get Python’s zlib.decompress to initialize– no luck. It’s frustrating to know that I’ll have to reverse engineer a compression algorithm that deals with just 8 total text files if I want to see this effort through to fruition.

  Update, January 15, 2019
  Some folks expressed interest in trying to sort out the details of the compression format. So I have posted a followup in which I post some samples and go into deeper details about things I have tried :

  Reverse Engineering Clue Chronicles Compression

  The post Parsing The Clue Chronicles first appeared on Breaking Eggs And Making Omelettes.

• How to stream synchronized video and audio in real-time from an Android smartphone using HLS while preserving orientation metadata ?

  6 mars, par Jérôme LAROSE

  Hello,  
I am working on an Android application where I need to stream video
from one or two cameras on my smartphone, along with audio from the
microphone, in real-time via a link or web page accessible to users.
The stream should be live, allow rewinding (DVR functionality), and be
recorded simultaneously. A latency of 1 to 2 minutes is acceptable,
and the streaming is one-way.  

I have chosen HLS (HTTP Live Streaming) for its browser compatibility
and DVR support. However, I am encountering issues with audio-video
synchronization, managing camera orientation metadata, and format
conversions.


  


  Here are my attempts :

  


  


  1. MP4 segmentation with MediaRecorder

     


     

     • I used MediaRecorder with setNextOutputFile to generate short MP4 segments, then ffmpeg-kit to convert them to fMP4 for HLS.

     


     • Expected : Well-aligned segments for smooth HLS playback.

     


     • Result : Timestamp issues causing jumps or interruptions in playback.

     


     


  


  2. MPEG2-TS via local socket

     


     

     • I configured MediaRecorder to produce an MPEG2-TS stream sent via a local socket to ffmpeg-kit.

     


     • Expected : Stable streaming with preserved metadata.

     


     • Result : Streaming works, but orientation metadata is lost, leading to incorrectly oriented video (e.g., rotated 90°).

     


     


  


  3. Orientation correction with ffmpeg

     


     

     • I tested -vf transpose=1 in ffmpeg to correct the orientation.

     


     • Expected : Correctly oriented video without excessive latency.

     


     • Result : Re-encoding takes too long for real-time streaming, causing unacceptable latency.

     


     


  


  4. MPEG2-TS to fMP4 conversion

     


     

     • I converted the MPEG2-TS stream to fMP4 with ffmpeg to preserve orientation.

     


     • Expected : Perfect audio-video synchronization.

     


     • Result : Slight desynchronization between audio and video, affecting the user experience.

     


     


  


  


  I am looking for a solution to :

  


  

  • Stream an HLS feed from Android with correctly timestamped segments.

  


  • Preserve orientation metadata without heavy re-encoding.

  


  • Ensure perfect audio-video synchronization.

  


  


  UPDATE

  


  package com.example.angegardien

import android.Manifest
import android.content.Context
import android.content.pm.PackageManager
import android.graphics.SurfaceTexture
import android.hardware.camera2.*
import android.media.*
import android.os.*
import android.util.Log
import android.view.Surface
import android.view.TextureView
import android.view.WindowManager
import androidx.activity.ComponentActivity
import androidx.core.app.ActivityCompat
import com.arthenica.ffmpegkit.FFmpegKit
import fi.iki.elonen.NanoHTTPD
import kotlinx.coroutines.*
import java.io.File
import java.io.IOException
import java.net.ServerSocket
import android.view.OrientationEventListener

/**
 * MainActivity class:
 * - Manages camera operations using the Camera2 API.
 * - Records video using MediaRecorder.
 * - Pipes data to FFmpeg to generate HLS segments.
 * - Hosts a local HLS server using NanoHTTPD to serve the generated HLS content.
 */
class MainActivity : ComponentActivity() {

    // TextureView used for displaying the camera preview.
    private lateinit var textureView: TextureView
    // Camera device instance.
    private lateinit var cameraDevice: CameraDevice
    // Camera capture session for managing capture requests.
    private lateinit var cameraCaptureSession: CameraCaptureSession
    // CameraManager to access camera devices.
    private lateinit var cameraManager: CameraManager
    // Directory where HLS output files will be stored.
    private lateinit var hlsDir: File
    // Instance of the HLS server.
    private lateinit var hlsServer: HlsServer

    // Camera id ("1" corresponds to the rear camera).
    private val cameraId = "1"
    // Flag indicating whether recording is currently active.
    private var isRecording = false

    // MediaRecorder used for capturing audio and video.
    private lateinit var activeRecorder: MediaRecorder
    // Surface for the camera preview.
    private lateinit var previewSurface: Surface
    // Surface provided by MediaRecorder for recording.
    private lateinit var recorderSurface: Surface

    // Port for the FFmpeg local socket connection.
    private val ffmpegPort = 8080

    // Coroutine scope to manage asynchronous tasks.
    private val scope = CoroutineScope(Dispatchers.IO + SupervisorJob())

    // Variables to track current device rotation and listen for orientation changes.
    private var currentRotation = 0
    private lateinit var orientationListener: OrientationEventListener

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)

        // Initialize the TextureView and set it as the content view.
        textureView = TextureView(this)
        setContentView(textureView)

        // Get the CameraManager system service.
        cameraManager = getSystemService(CAMERA_SERVICE) as CameraManager
        // Setup the directory for HLS output.
        setupHLSDirectory()

        // Start the local HLS server on port 8081.
        hlsServer = HlsServer(8081, hlsDir, this)
        try {
            hlsServer.start()
            Log.d("HLS_SERVER", "HLS Server started on port 8081")
        } catch (e: IOException) {
            Log.e("HLS_SERVER", "Error starting HLS Server", e)
        }

        // Initialize the current rotation.
        currentRotation = getDeviceRotation()

        // Add a listener to detect orientation changes.
        orientationListener = object : OrientationEventListener(this) {
            override fun onOrientationChanged(orientation: Int) {
                if (orientation == ORIENTATION_UNKNOWN) return // Skip unknown orientations.
                // Determine the new rotation angle.
                val newRotation = when {
                    orientation >= 315 || orientation < 45 -> 0
                    orientation >= 45 && orientation < 135 -> 90
                    orientation >= 135 && orientation < 225 -> 180
                    orientation >= 225 && orientation < 315 -> 270
                    else -> 0
                }
                // If the rotation has changed and recording is active, update the rotation.
                if (newRotation != currentRotation && isRecording) {
                    Log.d("ROTATION", "Orientation change detected: $newRotation")
                    currentRotation = newRotation
                }
            }
        }
        orientationListener.enable()

        // Set up the TextureView listener to know when the surface is available.
        textureView.surfaceTextureListener = object : TextureView.SurfaceTextureListener {
            override fun onSurfaceTextureAvailable(surface: SurfaceTexture, width: Int, height: Int) {
                // Open the camera when the texture becomes available.
                openCamera()
            }
            override fun onSurfaceTextureSizeChanged(surface: SurfaceTexture, width: Int, height: Int) {}
            override fun onSurfaceTextureDestroyed(surface: SurfaceTexture) = false
            override fun onSurfaceTextureUpdated(surface: SurfaceTexture) {}
        }
    }

    /**
     * Sets up the HLS directory in the public Downloads folder.
     * If the directory exists, it deletes it recursively and creates a new one.
     */
    private fun setupHLSDirectory() {
        val downloadsDir = Environment.getExternalStoragePublicDirectory(Environment.DIRECTORY_DOWNLOADS)
        hlsDir = File(downloadsDir, "HLS_Output")

        if (hlsDir.exists()) {
            hlsDir.deleteRecursively()
        }
        hlsDir.mkdirs()

        Log.d("HLS", "📂 HLS folder created: ${hlsDir.absolutePath}")
    }

    /**
     * Opens the camera after checking for necessary permissions.
     */
    private fun openCamera() {
        if (ActivityCompat.checkSelfPermission(this, Manifest.permission.CAMERA) != PackageManager.PERMISSION_GRANTED ||
            ActivityCompat.checkSelfPermission(this, Manifest.permission.RECORD_AUDIO) != PackageManager.PERMISSION_GRANTED) {
            // Request permissions if they are not already granted.
            ActivityCompat.requestPermissions(this, arrayOf(Manifest.permission.CAMERA, Manifest.permission.RECORD_AUDIO), 101)
            return
        }

        try {
            // Open the specified camera using its cameraId.
            cameraManager.openCamera(cameraId, object : CameraDevice.StateCallback() {
                override fun onOpened(camera: CameraDevice) {
                    cameraDevice = camera
                    // Start the recording session once the camera is opened.
                    startNextRecording()
                }
                override fun onDisconnected(camera: CameraDevice) { camera.close() }
                override fun onError(camera: CameraDevice, error: Int) { camera.close() }
            }, null)
        } catch (e: CameraAccessException) {
            e.printStackTrace()
        }
    }

    /**
     * Starts a new recording session:
     * - Sets up the preview and recorder surfaces.
     * - Creates a pipe for MediaRecorder output.
     * - Creates a capture session for simultaneous preview and recording.
     */
    private fun startNextRecording() {
        // Get the SurfaceTexture from the TextureView and set its default buffer size.
        val texture = textureView.surfaceTexture!!
        texture.setDefaultBufferSize(1920, 1080)
        // Create the preview surface.
        previewSurface = Surface(texture)

        // Create and configure the MediaRecorder.
        activeRecorder = createMediaRecorder()

        // Create a pipe to route MediaRecorder data.
        val pipe = ParcelFileDescriptor.createPipe()
        val pfdWrite = pipe[1] // Write end used by MediaRecorder.
        val pfdRead = pipe[0]  // Read end used by the local socket server.

        // Set MediaRecorder output to the file descriptor of the write end.
        activeRecorder.setOutputFile(pfdWrite.fileDescriptor)
        setupMediaRecorder(activeRecorder)
        // Obtain the recorder surface from MediaRecorder.
        recorderSurface = activeRecorder.surface

        // Create a capture request using the RECORD template.
        val captureRequestBuilder = cameraDevice.createCaptureRequest(CameraDevice.TEMPLATE_RECORD)
        captureRequestBuilder.addTarget(previewSurface)
        captureRequestBuilder.addTarget(recorderSurface)

        // Create a capture session including both preview and recorder surfaces.
        cameraDevice.createCaptureSession(
            listOf(previewSurface, recorderSurface),
            object : CameraCaptureSession.StateCallback() {
                override fun onConfigured(session: CameraCaptureSession) {
                    cameraCaptureSession = session
                    captureRequestBuilder.set(CaptureRequest.CONTROL_MODE, CameraMetadata.CONTROL_MODE_AUTO)
                    // Start a continuous capture request.
                    cameraCaptureSession.setRepeatingRequest(captureRequestBuilder.build(), null, null)

                    // Launch a coroutine to start FFmpeg and MediaRecorder with synchronization.
                    scope.launch {
                        startFFmpeg()
                        delay(500) // Wait for FFmpeg to be ready.
                        activeRecorder.start()
                        isRecording = true
                        Log.d("HLS", "🎥 Recording started...")
                    }

                    // Launch a coroutine to run the local socket server to forward data.
                    scope.launch {
                        startLocalSocketServer(pfdRead)
                    }
                }
                override fun onConfigureFailed(session: CameraCaptureSession) {
                    Log.e("Camera2", "❌ Configuration failed")
                }
            },
            null
        )
    }

    /**
     * Coroutine to start a local socket server.
     * It reads from the MediaRecorder pipe and sends the data to FFmpeg.
     */
    private suspend fun startLocalSocketServer(pfdRead: ParcelFileDescriptor) {
        withContext(Dispatchers.IO) {
            val serverSocket = ServerSocket(ffmpegPort)
            Log.d("HLS", "Local socket server started on port $ffmpegPort")

            // Accept connection from FFmpeg.
            val socket = serverSocket.accept()
            Log.d("HLS", "Connection accepted from FFmpeg")

            // Read data from the pipe and forward it through the socket.
            val inputStream = ParcelFileDescriptor.AutoCloseInputStream(pfdRead)
            val outputStream = socket.getOutputStream()
            val buffer = ByteArray(8192)
            var bytesRead: Int
            while (inputStream.read(buffer).also { bytesRead = it } != -1) {
                outputStream.write(buffer, 0, bytesRead)
            }
            outputStream.close()
            inputStream.close()
            socket.close()
            serverSocket.close()
        }
    }

    /**
     * Coroutine to start FFmpeg using a local TCP input.
     * Applies a video rotation filter based on device orientation and generates HLS segments.
     */
    private suspend fun startFFmpeg() {
        withContext(Dispatchers.IO) {
            // Retrieve the appropriate transpose filter based on current rotation.
            val transposeFilter = getTransposeFilter(currentRotation)

            // FFmpeg command to read from the TCP socket and generate an HLS stream.
            // Two alternative commands are commented below.
            // val ffmpegCommand = "-fflags +genpts -i tcp://localhost:$ffmpegPort -c copy -bsf:a aac_adtstoasc -movflags +faststart -f dash -seg_duration 10 -hls_playlist 1 ${hlsDir.absolutePath}/manifest.mpd"
            // val ffmpegCommand = "-fflags +genpts -i tcp://localhost:$ffmpegPort -c copy -bsf:a aac_adtstoasc -movflags +faststart -f hls -hls_time 5 -hls_segment_type fmp4 -hls_flags split_by_time -hls_list_size 0 -hls_playlist_type event -hls_fmp4_init_filename init.mp4 -hls_segment_filename ${hlsDir.absolutePath}/segment_%03d.m4s ${hlsDir.absolutePath}/playlist.m3u8"
            val ffmpegCommand = "-fflags +genpts -i tcp://localhost:$ffmpegPort -vf $transposeFilter -c:v libx264 -preset ultrafast -crf 23 -c:a copy -movflags +faststart -f hls -hls_time 0.1 -hls_segment_type mpegts -hls_flags split_by_time -hls_list_size 0 -hls_playlist_type event -hls_segment_filename ${hlsDir.absolutePath}/segment_%03d.ts ${hlsDir.absolutePath}/playlist.m3u8"

            FFmpegKit.executeAsync(ffmpegCommand) { session ->
                if (session.returnCode.isValueSuccess) {
                    Log.d("HLS", "✅ HLS generated successfully")
                } else {
                    Log.e("FFmpeg", "❌ Error generating HLS: ${session.allLogsAsString}")
                }
            }
        }
    }

    /**
     * Gets the current device rotation using the WindowManager.
     */
    private fun getDeviceRotation(): Int {
        val windowManager = getSystemService(Context.WINDOW_SERVICE) as WindowManager
        return when (windowManager.defaultDisplay.rotation) {
            Surface.ROTATION_0 -> 0
            Surface.ROTATION_90 -> 90
            Surface.ROTATION_180 -> 180
            Surface.ROTATION_270 -> 270
            else -> 0
        }
    }

    /**
     * Returns the FFmpeg transpose filter based on the rotation angle.
     * Used to rotate the video stream accordingly.
     */
    private fun getTransposeFilter(rotation: Int): String {
        return when (rotation) {
            90 -> "transpose=1" // 90° clockwise
            180 -> "transpose=2,transpose=2" // 180° rotation
            270 -> "transpose=2" // 90° counter-clockwise
            else -> "transpose=0" // No rotation
        }
    }

    /**
     * Creates and configures a MediaRecorder instance.
     * Sets up audio and video sources, formats, encoders, and bitrates.
     */
    private fun createMediaRecorder(): MediaRecorder {
        return MediaRecorder().apply {
            setAudioSource(MediaRecorder.AudioSource.MIC)
            setVideoSource(MediaRecorder.VideoSource.SURFACE)
            setOutputFormat(MediaRecorder.OutputFormat.MPEG_2_TS)
            setVideoEncodingBitRate(5000000)
            setVideoFrameRate(24)
            setVideoSize(1080, 720)
            setVideoEncoder(MediaRecorder.VideoEncoder.H264)
            setAudioEncoder(MediaRecorder.AudioEncoder.AAC)
            setAudioSamplingRate(16000)
            setAudioEncodingBitRate(96000) // 96 kbps
        }
    }

    /**
     * Prepares the MediaRecorder and logs the outcome.
     */
    private fun setupMediaRecorder(recorder: MediaRecorder) {
        try {
            recorder.prepare()
            Log.d("HLS", "✅ MediaRecorder prepared")
        } catch (e: IOException) {
            Log.e("HLS", "❌ Error preparing MediaRecorder", e)
        }
    }

    /**
     * Custom HLS server class extending NanoHTTPD.
     * Serves HLS segments and playlists from the designated HLS directory.
     */
    private inner class HlsServer(port: Int, private val hlsDir: File, private val context: Context) : NanoHTTPD(port) {
        override fun serve(session: IHTTPSession): Response {
            val uri = session.uri.trimStart('/')

            // Intercept the request for `init.mp4` and serve it from assets.
            /*
            if (uri == "init.mp4") {
                Log.d("HLS Server", "📡 Intercepting init.mp4, sending file from assets...")
                return try {
                    val assetManager = context.assets
                    val inputStream = assetManager.open("init.mp4")
                    newFixedLengthResponse(Response.Status.OK, "video/mp4", inputStream, inputStream.available().toLong())
                } catch (e: Exception) {
                    Log.e("HLS Server", "❌ Error reading init.mp4 from assets: ${e.message}")
                    newFixedLengthResponse(Response.Status.INTERNAL_ERROR, MIME_PLAINTEXT, "Server error")
                }
            }
            */

            // Serve all other HLS files normally from the hlsDir.
            val file = File(hlsDir, uri)
            return if (file.exists()) {
                newFixedLengthResponse(Response.Status.OK, getMimeTypeForFile(uri), file.inputStream(), file.length())
            } else {
                newFixedLengthResponse(Response.Status.NOT_FOUND, MIME_PLAINTEXT, "File not found")
            }
        }
    }

    /**
     * Clean up resources when the activity is destroyed.
     * Stops recording, releases the camera, cancels coroutines, and stops the HLS server.
     */
    override fun onDestroy() {
        super.onDestroy()
        if (isRecording) {
            activeRecorder.stop()
            activeRecorder.release()
        }
        cameraDevice.close()
        scope.cancel()
        hlsServer.stop()
        orientationListener.disable()
        Log.d("HLS", "🛑 Activity destroyed")
    }
}


  


  I have three examples of ffmpeg commands.

  


  

  • One command segments into DASH, but the camera does not have the correct rotation.

  


  • One command segments into HLS without re-encoding with 5-second segments ; it’s fast but does not have the correct rotation.

  


  • One command segments into HLS with re-encoding, which applies a rotation. It’s too slow for 5-second segments, so a 1-second segment was chosen.

  


  


  Note :

  


  

  • In the second command ("One command segments into HLS without re-encoding with 5-second segments ; it’s fast but does not have the correct rotation."), it returns fMP4. To achieve the correct rotation, I provide a preconfigured init.mp4 file during the HTTP request to retrieve it (see comment).

  


  • In the third command ("One command segments into HLS with re-encoding, which applies a rotation. It’s too slow for 5-second segments, so a 1-second segment was chosen."), it returns TS.

  


  


• C++ ffmpeg lib version 7.0 - distortion in exported audio

  4 septembre 2024, par Chris P

  I want to make a C++ lib named cppdub which will mimic the python module pydub.

  


  One main function is to export the AudioSegment to a file with a specific format (example : mp3).

  


  The code is :

  


  &#xA;AudioSegment AudioSegment::from_file(const std::string&amp; file_path, const std::string&amp; format, const std::string&amp; codec,&#xA;    const std::map&amp; parameters, int start_second, int duration) {&#xA;&#xA;    avformat_network_init();&#xA;    av_log_set_level(AV_LOG_ERROR); // Adjust logging level as needed&#xA;&#xA;    AVFormatContext* format_ctx = nullptr;&#xA;    if (avformat_open_input(&amp;format_ctx, file_path.c_str(), nullptr, nullptr) != 0) {&#xA;        std::cerr &lt;&lt; "Error: Could not open audio file." &lt;&lt; std::endl;&#xA;        return AudioSegment();  // Return an empty AudioSegment on failure&#xA;    }&#xA;&#xA;    if (avformat_find_stream_info(format_ctx, nullptr) &lt; 0) {&#xA;        std::cerr &lt;&lt; "Error: Could not find stream information." &lt;&lt; std::endl;&#xA;        avformat_close_input(&amp;format_ctx);&#xA;        return AudioSegment();&#xA;    }&#xA;&#xA;    int audio_stream_index = -1;&#xA;    for (unsigned int i = 0; i &lt; format_ctx->nb_streams; i&#x2B;&#x2B;) {&#xA;        if (format_ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) {&#xA;            audio_stream_index = i;&#xA;            break;&#xA;        }&#xA;    }&#xA;&#xA;    if (audio_stream_index == -1) {&#xA;        std::cerr &lt;&lt; "Error: Could not find audio stream." &lt;&lt; std::endl;&#xA;        avformat_close_input(&amp;format_ctx);&#xA;        return AudioSegment();&#xA;    }&#xA;&#xA;    AVCodecParameters* codec_par = format_ctx->streams[audio_stream_index]->codecpar;&#xA;    const AVCodec* my_codec = avcodec_find_decoder(codec_par->codec_id);&#xA;    AVCodecContext* codec_ctx = avcodec_alloc_context3(my_codec);&#xA;&#xA;    if (!codec_ctx) {&#xA;        std::cerr &lt;&lt; "Error: Could not allocate codec context." &lt;&lt; std::endl;&#xA;        avformat_close_input(&amp;format_ctx);&#xA;        return AudioSegment();&#xA;    }&#xA;&#xA;    if (avcodec_parameters_to_context(codec_ctx, codec_par) &lt; 0) {&#xA;        std::cerr &lt;&lt; "Error: Could not initialize codec context." &lt;&lt; std::endl;&#xA;        avcodec_free_context(&amp;codec_ctx);&#xA;        avformat_close_input(&amp;format_ctx);&#xA;        return AudioSegment();&#xA;    }&#xA;&#xA;    if (avcodec_open2(codec_ctx, my_codec, nullptr) &lt; 0) {&#xA;        std::cerr &lt;&lt; "Error: Could not open codec." &lt;&lt; std::endl;&#xA;        avcodec_free_context(&amp;codec_ctx);&#xA;        avformat_close_input(&amp;format_ctx);&#xA;        return AudioSegment();&#xA;    }&#xA;&#xA;    SwrContext* swr_ctx = swr_alloc();&#xA;    if (!swr_ctx) {&#xA;        std::cerr &lt;&lt; "Error: Could not allocate SwrContext." &lt;&lt; std::endl;&#xA;        avcodec_free_context(&amp;codec_ctx);&#xA;        avformat_close_input(&amp;format_ctx);&#xA;        return AudioSegment();&#xA;    }&#xA;&#xA;    // Set up resampling context to convert to S16 format with 2 bytes per sample&#xA;    av_opt_set_chlayout(swr_ctx, "in_chlayout", &amp;codec_ctx->ch_layout, 0);&#xA;    av_opt_set_int(swr_ctx, "in_sample_rate", codec_ctx->sample_rate, 0);&#xA;    av_opt_set_sample_fmt(swr_ctx, "in_sample_fmt", codec_ctx->sample_fmt, 0);&#xA;&#xA;    AVChannelLayout dst_ch_layout;&#xA;    av_channel_layout_copy(&amp;dst_ch_layout, &amp;codec_ctx->ch_layout);&#xA;    av_channel_layout_uninit(&amp;dst_ch_layout);&#xA;    av_channel_layout_default(&amp;dst_ch_layout, 2);&#xA;&#xA;    av_opt_set_chlayout(swr_ctx, "out_chlayout", &amp;dst_ch_layout, 0);&#xA;    av_opt_set_int(swr_ctx, "out_sample_rate", codec_ctx->sample_rate, 0);  // Match input sample rate&#xA;    av_opt_set_sample_fmt(swr_ctx, "out_sample_fmt", AV_SAMPLE_FMT_S16, 0);  // Force S16 format&#xA;&#xA;    if (swr_init(swr_ctx) &lt; 0) {&#xA;        std::cerr &lt;&lt; "Error: Failed to initialize the resampling context" &lt;&lt; std::endl;&#xA;        swr_free(&amp;swr_ctx);&#xA;        avcodec_free_context(&amp;codec_ctx);&#xA;        avformat_close_input(&amp;format_ctx);&#xA;        return AudioSegment();&#xA;    }&#xA;&#xA;    AVPacket packet;&#xA;    AVFrame* frame = av_frame_alloc();&#xA;    if (!frame) {&#xA;        std::cerr &lt;&lt; "Error: Could not allocate frame." &lt;&lt; std::endl;&#xA;        swr_free(&amp;swr_ctx);&#xA;        avcodec_free_context(&amp;codec_ctx);&#xA;        avformat_close_input(&amp;format_ctx);&#xA;        return AudioSegment();&#xA;    }&#xA;&#xA;    std::vector<char> output;&#xA;    while (av_read_frame(format_ctx, &amp;packet) >= 0) {&#xA;        if (packet.stream_index == audio_stream_index) {&#xA;            if (avcodec_send_packet(codec_ctx, &amp;packet) == 0) {&#xA;                while (avcodec_receive_frame(codec_ctx, frame) == 0) {&#xA;                    if (frame->pts != AV_NOPTS_VALUE) {&#xA;                        frame->pts = av_rescale_q(frame->pts, codec_ctx->time_base, format_ctx->streams[audio_stream_index]->time_base);&#xA;                    }&#xA;&#xA;                    uint8_t* output_buffer;&#xA;                    int output_samples = av_rescale_rnd(&#xA;                        swr_get_delay(swr_ctx, codec_ctx->sample_rate) &#x2B; frame->nb_samples,&#xA;                        codec_ctx->sample_rate, codec_ctx->sample_rate, AV_ROUND_UP);&#xA;&#xA;                    int output_buffer_size = av_samples_get_buffer_size(&#xA;                        nullptr, 2, output_samples, AV_SAMPLE_FMT_S16, 1);&#xA;&#xA;                    output_buffer = (uint8_t*)av_malloc(output_buffer_size);&#xA;&#xA;                    if (output_buffer) {&#xA;                        memset(output_buffer, 0, output_buffer_size); // Zero padding to avoid random noise&#xA;                        int converted_samples = swr_convert(swr_ctx, &amp;output_buffer, output_samples,&#xA;                            (const uint8_t**)frame->extended_data, frame->nb_samples);&#xA;&#xA;                        if (converted_samples >= 0) {&#xA;                            output.insert(output.end(), output_buffer, output_buffer &#x2B; output_buffer_size);&#xA;                        }&#xA;                        else {&#xA;                            std::cerr &lt;&lt; "Error: Failed to convert audio samples." &lt;&lt; std::endl;&#xA;                        }&#xA;                        // Make sure output_buffer is valid before freeing&#xA;                        if (output_buffer != nullptr) {&#xA;                            av_free(output_buffer);&#xA;                            output_buffer = nullptr; // Prevent double-free&#xA;                        }&#xA;                    }&#xA;                    else {&#xA;                        std::cerr &lt;&lt; "Error: Could not allocate output buffer." &lt;&lt; std::endl;&#xA;                    }&#xA;                }&#xA;            }&#xA;            else {&#xA;                std::cerr &lt;&lt; "Error: Failed to send packet to codec context." &lt;&lt; std::endl;&#xA;            }&#xA;        }&#xA;        av_packet_unref(&amp;packet);&#xA;    }&#xA;&#xA;    int frame_width = av_get_bytes_per_sample(AV_SAMPLE_FMT_S16) * 2;  // Use 2 bytes per sample and 2 channels&#xA;&#xA;    std::map metadata = {&#xA;        {"sample_width", 2},  // S16 format has 2 bytes per sample&#xA;        {"frame_rate", codec_ctx->sample_rate},  // Use the input sample rate&#xA;        {"channels", 2},  // Assuming stereo output&#xA;        {"frame_width", frame_width}&#xA;    };&#xA;&#xA;    av_frame_free(&amp;frame);&#xA;    swr_free(&amp;swr_ctx);&#xA;    avcodec_free_context(&amp;codec_ctx);&#xA;    avformat_close_input(&amp;format_ctx);&#xA;&#xA;    return AudioSegment(static_cast<const>(output.data()), output.size(), metadata);&#xA;}&#xA;&#xA;&#xA;&#xA;&#xA;std::ofstream AudioSegment::export_segment(const std::string&amp; out_f,&#xA;    const std::string&amp; format,&#xA;    const std::string&amp; codec,&#xA;    const std::string&amp; bitrate,&#xA;    const std::vector&amp; parameters,&#xA;    const std::map&amp; tags,&#xA;    const std::string&amp; id3v2_version,&#xA;    const std::string&amp; cover) {&#xA;    av_log_set_level(AV_LOG_DEBUG);&#xA;    AVCodecContext* codec_ctx = nullptr;&#xA;    AVFormatContext* format_ctx = nullptr;&#xA;    AVStream* stream = nullptr;&#xA;    AVFrame* frame = nullptr;&#xA;    AVPacket* pkt = nullptr;&#xA;    SwrContext* swr_ctx = swr_alloc();&#xA;    int ret;&#xA;&#xA;    // Initialize format context&#xA;    if (avformat_alloc_output_context2(&amp;format_ctx, nullptr, format.c_str(), out_f.c_str()) &lt; 0) {&#xA;        throw std::runtime_error("Could not allocate format context.");&#xA;    }&#xA;&#xA;    // Find encoder&#xA;    const AVCodec* codec_ptr = avcodec_find_encoder_by_name(codec.c_str());&#xA;    if (!codec_ptr) {&#xA;        throw std::runtime_error("Codec not found.");&#xA;    }&#xA;&#xA;    // Add stream&#xA;    stream = avformat_new_stream(format_ctx, codec_ptr);&#xA;    if (!stream) {&#xA;        throw std::runtime_error("Failed to create new stream.");&#xA;    }&#xA;&#xA;    // Allocate codec context&#xA;    codec_ctx = avcodec_alloc_context3(codec_ptr);&#xA;    if (!codec_ctx) {&#xA;        throw std::runtime_error("Could not allocate audio codec context.");&#xA;    }&#xA;&#xA;    // Set codec parameters&#xA;    codec_ctx->bit_rate = std::stoi(bitrate);&#xA;    codec_ctx->sample_rate = this->get_frame_rate(); // Assuming get_frame_rate() returns the correct sample rate&#xA;&#xA;    // Set channel layout for stereo output&#xA;    av_channel_layout_default(&amp;codec_ctx->ch_layout, 2);&#xA;    codec_ctx->sample_fmt = AV_SAMPLE_FMT_S16P;&#xA;&#xA;    // Open codec&#xA;    if (avcodec_open2(codec_ctx, codec_ptr, nullptr) &lt; 0) {&#xA;        throw std::runtime_error("Could not open codec.");&#xA;    }&#xA;&#xA;    // Set codec parameters to the stream&#xA;    if (avcodec_parameters_from_context(stream->codecpar, codec_ctx) &lt; 0) {&#xA;        throw std::runtime_error("Could not initialize stream codec parameters.");&#xA;    }&#xA;&#xA;    // Open output file&#xA;    std::ofstream out_file(out_f, std::ios::binary);&#xA;    if (!out_file) {&#xA;        throw std::runtime_error("Failed to open output file.");&#xA;    }&#xA;&#xA;    if (!(format_ctx->oformat->flags &amp; AVFMT_NOFILE)) {&#xA;        if (avio_open(&amp;format_ctx->pb, out_f.c_str(), AVIO_FLAG_WRITE) &lt; 0) {&#xA;            throw std::runtime_error("Could not open output file.");&#xA;        }&#xA;    }&#xA;&#xA;    // Write file header&#xA;    if (avformat_write_header(format_ctx, nullptr) &lt; 0) {&#xA;        throw std::runtime_error("Error occurred when opening output file.");&#xA;    }&#xA;&#xA;    // Initialize packet&#xA;    pkt = av_packet_alloc();&#xA;    if (!pkt) {&#xA;        throw std::runtime_error("Could not allocate AVPacket.");&#xA;    }&#xA;&#xA;    // Initialize frame&#xA;    frame = av_frame_alloc();&#xA;    if (!frame) {&#xA;        throw std::runtime_error("Could not allocate AVFrame.");&#xA;    }&#xA;    frame->nb_samples = codec_ctx->frame_size;&#xA;    frame->format = codec_ctx->sample_fmt;&#xA;    frame->ch_layout = codec_ctx->ch_layout;&#xA;&#xA;    // Allocate data buffer&#xA;    if (av_frame_get_buffer(frame, 0) &lt; 0) {&#xA;        throw std::runtime_error("Could not allocate audio data buffers.");&#xA;    }&#xA;&#xA;    // Initialize SwrContext for resampling&#xA;    if (!swr_ctx) {&#xA;        throw std::runtime_error("Could not allocate SwrContext.");&#xA;    }&#xA;&#xA;    // Set input and output options&#xA;    av_opt_set_chlayout(swr_ctx, "in_chlayout", &amp;codec_ctx->ch_layout, 0);&#xA;    av_opt_set_chlayout(swr_ctx, "out_chlayout", &amp;codec_ctx->ch_layout, 0);&#xA;    av_opt_set_int(swr_ctx, "in_sample_rate", codec_ctx->sample_rate, 0);&#xA;    av_opt_set_int(swr_ctx, "out_sample_rate", codec_ctx->sample_rate, 0);&#xA;    av_opt_set_sample_fmt(swr_ctx, "in_sample_fmt", codec_ctx->sample_fmt, 0);&#xA;    av_opt_set_sample_fmt(swr_ctx, "out_sample_fmt", codec_ctx->sample_fmt, 0);&#xA;&#xA;    // Initialize the resampling context&#xA;    if (swr_init(swr_ctx) &lt; 0) {&#xA;        throw std::runtime_error("Failed to initialize SwrContext.");&#xA;    }&#xA;&#xA;    // Allocate buffer for resampled data&#xA;    int samples_read = 0;&#xA;    int total_samples = data_.size() / (av_get_bytes_per_sample(codec_ctx->sample_fmt) * codec_ctx->ch_layout.nb_channels);&#xA;&#xA;    int num_channels = codec_ctx->ch_layout.nb_channels;&#xA;    int bytes_per_sample = av_get_bytes_per_sample(codec_ctx->sample_fmt);&#xA;    int buffer_size = codec_ctx->frame_size * num_channels * bytes_per_sample;&#xA;    uint8_t* resampled_data = (uint8_t*)av_malloc(buffer_size);&#xA;    if (!resampled_data) {&#xA;        throw std::runtime_error("Could not allocate buffer for resampled data.");&#xA;    }&#xA;&#xA;    // Set up buffer pointers for swr_convert&#xA;    uint8_t* resampled_data_array[2] = { nullptr, nullptr };&#xA;    for (int i = 0; i &lt; num_channels; &#x2B;&#x2B;i) {&#xA;        resampled_data_array[i] = resampled_data &#x2B; i * codec_ctx->frame_size * bytes_per_sample;&#xA;    }&#xA;&#xA;    while (samples_read &lt; total_samples) {&#xA;        if (av_frame_make_writable(frame) &lt; 0) {&#xA;            throw std::runtime_error("Frame not writable.");&#xA;        }&#xA;&#xA;        int num_samples = std::min(codec_ctx->frame_size, total_samples - samples_read);&#xA;&#xA;        if (av_sample_fmt_is_planar(codec_ctx->sample_fmt)) {&#xA;            for (int ch = 0; ch &lt; num_channels; &#x2B;&#x2B;ch) {&#xA;                int channel_size = num_samples * bytes_per_sample;&#xA;                std::memcpy(frame->data[ch],&#xA;                    data_.data() &#x2B; (samples_read * bytes_per_sample * num_channels) &#x2B; (ch * channel_size),&#xA;                    channel_size);&#xA;            }&#xA;        }&#xA;        else {&#xA;            int buffer_size = num_samples * bytes_per_sample * num_channels;&#xA;            std::memcpy(frame->data[0],&#xA;                data_.data() &#x2B; samples_read * bytes_per_sample * num_channels,&#xA;                buffer_size);&#xA;        }&#xA;&#xA;        // Resample audio data&#xA;        int output_samples = av_rescale_rnd(swr_get_delay(swr_ctx, codec_ctx->sample_rate) &#x2B; frame->nb_samples,&#xA;            codec_ctx->sample_rate, codec_ctx->sample_rate, AV_ROUND_UP);&#xA;        int converted_samples = swr_convert(swr_ctx, resampled_data_array, output_samples,&#xA;            (const uint8_t**)frame->data, frame->nb_samples);&#xA;&#xA;        if (converted_samples &lt; 0) {&#xA;            av_free(resampled_data);&#xA;            throw std::runtime_error("Error converting audio samples.");&#xA;        }&#xA;&#xA;        // Send the frame for encoding&#xA;        if (avcodec_send_frame(codec_ctx, frame) &lt; 0) {&#xA;            av_free(resampled_data);&#xA;            throw std::runtime_error("Error sending frame for encoding.");&#xA;        }&#xA;&#xA;        // Receive and write packets&#xA;        while (avcodec_receive_packet(codec_ctx, pkt) >= 0) {&#xA;            out_file.write(reinterpret_cast(pkt->data), pkt->size);&#xA;            av_packet_unref(pkt);&#xA;        }&#xA;&#xA;        samples_read &#x2B;= num_samples;&#xA;&#xA;        // If the frame is partially filled, pad the remaining part with zeros&#xA;        if (num_samples &lt; codec_ctx->frame_size) {&#xA;            for (int ch = 0; ch &lt; num_channels; &#x2B;&#x2B;ch) {&#xA;                int padding_size = (codec_ctx->frame_size - num_samples) * bytes_per_sample;&#xA;                std::memset(frame->data[ch] &#x2B; num_samples * bytes_per_sample, 0, padding_size);&#xA;            }&#xA;        }&#xA;    }&#xA;&#xA;    // Flush the encoder&#xA;    if (avcodec_send_frame(codec_ctx, nullptr) &lt; 0) {&#xA;        av_free(resampled_data);&#xA;        throw std::runtime_error("Error flushing the encoder.");&#xA;    }&#xA;&#xA;    while (avcodec_receive_packet(codec_ctx, pkt) >= 0) {&#xA;        out_file.write(reinterpret_cast(pkt->data), pkt->size);&#xA;        av_packet_unref(pkt);&#xA;    }&#xA;&#xA;    // Write file trailer&#xA;    av_write_trailer(format_ctx);&#xA;&#xA;    // Cleanup&#xA;    av_frame_free(&amp;frame);&#xA;    av_packet_free(&amp;pkt);&#xA;    av_free(resampled_data);&#xA;    swr_free(&amp;swr_ctx);&#xA;    avcodec_free_context(&amp;codec_ctx);&#xA;&#xA;    if (!(format_ctx->oformat->flags &amp; AVFMT_NOFILE)) {&#xA;        avio_closep(&amp;format_ctx->pb);&#xA;    }&#xA;    avformat_free_context(format_ctx);&#xA;&#xA;    out_file.close();&#xA;    return out_file;&#xA;}&#xA;&#xA;&#xA;</const></char>

  


  I have no run time error but i see this message in console :

  


  [file @ 0000025906626100] Setting default whitelist 'file,crypto,data'
[SWR @ 0000025906632040] Using s16p internally between filters
[libmp3lame @ 0000025906609b00] Trying to remove 47 more samples than there are in the queue
[AVIOContext @ 0000025906608540] Statistics: 566 bytes written, 0 seeks, 1 writeouts


  


  I can play the exported mp3 file but there is sound distortion.