Recherche avancée
Médias (3)
-
Elephants Dream - Cover of the soundtrack
17 octobre 2011, par
Mis à jour : Octobre 2011
Langue : English
Type : Image
-
Valkaama DVD Label
4 octobre 2011, par
Mis à jour : Février 2013
Langue : English
Type : Image
-
Publier une image simplement
13 avril 2011, par ,
Mis à jour : Février 2012
Langue : français
Type : Video
Autres articles (65)
-
Gestion générale des documents
13 mai 2011, parMédiaSPIP ne modifie jamais le document original mis en ligne.
Pour chaque document mis en ligne il effectue deux opérations successives : la création d’une version supplémentaire qui peut être facilement consultée en ligne tout en laissant l’original téléchargeable dans le cas où le document original ne peut être lu dans un navigateur Internet ; la récupération des métadonnées du document original pour illustrer textuellement le fichier ;
Les tableaux ci-dessous expliquent ce que peut faire MédiaSPIP (...) -
Des sites réalisés avec MediaSPIP
2 mai 2011, parCette page présente quelques-uns des sites fonctionnant sous MediaSPIP.
Vous pouvez bien entendu ajouter le votre grâce au formulaire en bas de page. -
HTML5 audio and video support
13 avril 2011, parMediaSPIP uses HTML5 video and audio tags to play multimedia files, taking advantage of the latest W3C innovations supported by modern browsers.
The MediaSPIP player used has been created specifically for MediaSPIP and can be easily adapted to fit in with a specific theme.
For older browsers the Flowplayer flash fallback is used.
MediaSPIP allows for media playback on major mobile platforms with the above (...)
Sur d’autres sites (6579)
-
Code can not read property 1 of undefined [closed]
25 mai 2023, par Jesse CopasI'm a very new programmer and am working on a Tdarr plugin in JS.
Everything works fine until a 4k file tries to get transcoded and it fails with this log


2023-05-24T19:09:54.906Z ZoBKWMMKG:Node\[hidden-hog\]:Worker\[tall-tuna\]:{"pluginInputs":{"BitRate":"4000","ResolutionSelection":"1080p","Container":"mkv","AudioType":"AAC","FrameRate":"24"}}

2023-05-24T19:09:54.907Z ZoBKWMMKG:Node\[hidden-hog\]:Worker\[tall-tuna\]:Error TypeError: Cannot read property '1' of undefined


It's saying that it's unable to read property 1 of undefined and I'm looked and looked and looked and can't find what it is referring to. Hoping to get another set of eyes on it
The plugin Code is here


/* eslint no-plusplus: ["error", { "allowForLoopAfterthoughts": true }] */
/* eslint-disable no-restricted-globals */
const details = () => ({
 id: 'Tdarr_Plugin_Jeso_AV1_HandBrake_Transcode',
 Stage: 'Pre-processing',
 Name: 'AV1 HandBrake Transcoder',
 Type: 'Video',
 Operation: 'Transcode',
 Description: 'Transcodes to AV1 at the selected Bitrate. This is best used with Remux Files.',
 Version: '2.1.3',
 Tags: 'HandBrake,configurable',
 Inputs: [
 {
 name: 'BitRate',
 type: 'string',
 defaultValue: '4000',
 inputUI: {
 type: 'text',
 },
 tooltip: `
 ~ Requested Bitrate ~ \\n
 Put in the Bitrate you want to process to in Kbps. For example 4000Kbps is 4Mbps. `,
 },
 {
 name: 'ResolutionSelection',
 type: 'string',
 defaultValue: '1080p',
 inputUI: {
 type: 'dropdown',
 options: [
 '8KUHD',
 '4KUHD',
 '1080p',
 '720p',
 '480p',
 ],
 },
 // eslint-disable-next-line max-len
 tooltip: 'Any Resolution larger than this will become this Resolution same as the bitrate if the Res is lower than the selected it will use the res of the file as to not cause bloating of file size.',
 },
 {
 name: 'Container',
 type: 'string',
 defaultValue: 'mkv',
 inputUI: {
 type: 'dropdown',
 options: [
 'mp4',
 'mkv',
 ],
 },
 tooltip: ` Container Type \\n\\n
 mkv or mp4.\\n`,
 },
 {
 name: 'AudioType',
 type: 'string',
 defaultValue: 'AAC',
 inputUI: {
 type: 'dropdown',
 options: [
 'AAC',
 'EAC3',
 'MP3',
 'Vorbis',
 'Flac16',
 'Flac24',
 ],
 },
 // eslint-disable-next-line max-len
 tooltip: 'Set Audio container type that you want to use',
 },
 {
 name: 'FrameRate',
 type: 'string',
 defaultValue: '24',
 inputUI: {
 type: 'text',
 },
 // eslint-disable-next-line max-len
 tooltip: 'If the files framerate is higher than 24 and you want to maintain that framerate you can do so here',
 },
 ],
});
const MediaInfo = {
 videoHeight: '',
 videoWidth: '',
 videoFPS: '',
 videoBR: '',
 videoBitDepth: '',
 overallBR: '',
 videoResolution: '',
}; // var MediaInfo
// Easier for our functions if response has global scope.
const response = {
 processFile: false,
 preset: '',
 container: '.mkv',
 handBrakeMode: true,
 FFmpegMode: false,
 reQueueAfter: true,
 infoLog: '',
}; // var response
// Finds the first video stream and populates some useful variables
function getMediaInfo(file) {
 let videoIdx = -1;
 for (let i = 0; i < file.ffProbeData.streams.length; i += 1) {
 const strstreamType = file.ffProbeData.streams[i].codec_type.toLowerCase();
 // Looking For Video
 // Check if stream is a video.
 if (videoIdx === -1 && strstreamType === 'video') {
 videoIdx = i;
 // get video streams resolution
 MediaInfo.videoResolution = `${file.ffProbeData.streams[i].height}x${file.ffProbeData.streams[i].width}`;
 MediaInfo.videoHeight = Number(file.ffProbeData.streams[i].height);
 MediaInfo.videoWidth = Number(file.ffProbeData.streams[i].width);
 MediaInfo.videoFPS = Number(file.mediaInfo.track[i + 1].FrameRate) || 25;
 // calulate bitrate from dimensions and fps of file
 MediaInfo.videoBR = (MediaInfo.videoHeight * MediaInfo.videoWidth * MediaInfo.videoFPS * 0.08).toFixed(0);
 }
 }
} // end getMediaInfo()
// define resolution order from ResolutionSelection from biggest to smallest
const resolutionOrder = ['8KUHD', '4KUHD', '1080p', '720p', '480p'];
// define the width and height of each resolution from the resolution order
const resolutionsdimensions = {
 '8KUHD': '--width 7680 --height 4320',
 '4KUHD': '--width 3840 --height 2160',
 '1080p': '--width 1920 --height 1080',
 '720p': '--width 1280 --height 720',
 '480p': '--width 640 --height 480',
};
// eslint-disable-next-line no-unused-vars
const plugin = (file, librarySettings, inputs) => {
 // eslint-disable-next-line no-unused-vars
 const importFresh = require('import-fresh');
 // eslint-disable-next-line no-unused-vars
 const library = importFresh('../methods/library.js');
 // eslint-disable-next-line no-unused-vars
 const lib = require('../methods/lib')();
 // Get the selected resolution from the 'ResolutionSelection' variable
 const selectedResolution = inputs.ResolutionSelection;
 getMediaInfo(file);
 // use mediainfo to match height and width to a resolution on resolutiondimensions
 let dimensions = resolutionsdimensions[selectedResolution];
 // if the file is smaller than the selected resolution then use the file resolution
 if (MediaInfo.videoHeight < dimensions.split(' ')[3] || MediaInfo.videoWidth < dimensions.split(' ')[1]) {
 dimensions = `--width ${MediaInfo.videoWidth} --height ${MediaInfo.videoHeight}`;
 // eslint-disable-next-line brace-style
 }
 // read the bitrate of the video stream
 let videoBitRate = MediaInfo.videoBR;
 // if videoBitrate is over 1000000 devide by 100 to get the bitrate in Kbps
 if (videoBitRate > 1000000) {
 videoBitRate /= 100;
 } else { videoBitRate /= 1000; }
 // if VideoBitrate is smaller than selected bitrate then use the videoBitrate
 if (videoBitRate < inputs.BitRate) {
 // eslint-disable-next-line no-param-reassign
 inputs.BitRate = videoBitRate;
 // eslint-disable-next-line brace-style
 }
 // if VideoBitrate is larger than selected bitrate then use the selected bitrate
 else {
 // eslint-disable-next-line no-self-assign, no-param-reassign
 inputs.BitRate = inputs.BitRate;
 }

 //Skip Transcoding if File is already AV1
 if (file.ffProbeData.streams[0].codec_name === 'av1') {
 response.processFile = false;
 response.infoLog += 'File is already AV1 \n';
 return response;
 }
 // eslint-disable-next-line no-constant-condition
 if ((true) || file.forceProcessing === true) {
 // eslint-disable-next-line max-len
 response.preset = `--encoder svt_av1 -b ${inputs.BitRate} -r ${inputs.FrameRate} -E ${inputs.AudioType} -f ${inputs.Container} --no-optimize ${dimensions} --crop 0:0:0:0`;
 response.container = `.${inputs.Container}`;
 response.handbrakeMode = true;
 response.ffmpegMode = false;
 response.processFile = true;
 response.infoLog += `File is being transcoded at ${inputs.BitRate} Kbps to ${dimensions} as ${inputs.Container} \n`;
 return response;
 }
 response.infoLog += 'File is being transcoded using custom arguments \n';
 return response;
};
 };

module.exports.details = details;
module.exports.plugin = plugin;


Tried transcoding 4k files down to 1080p but it fails due to that undefined error. All Res 1080p and lower that I have tried work correctly


EDIT : I used Console.log and got this back


[2023-05-24T23:29:51.001] [ERROR] Tdarr_Server - Error running MediaInfo 1
[2023-05-24T23:29:51.004] [ERROR] Tdarr_Server - RangeError: Maximum call stack size exceeded
 at x (<anonymous>:wasm-function[381]:0x15c4d)
 at <anonymous>:wasm-function[46]:0x5dc0
 at <anonymous>:wasm-function[652]:0x21cb9
 at <anonymous>:wasm-function[1023]:0x47018
 at <anonymous>:wasm-function[853]:0x37827
 at <anonymous>:wasm-function[3684]:0xf4884
 at <anonymous>:wasm-function[3516]:0xeb5b7
 at <anonymous>:wasm-function[1061]:0x487c9
 at <anonymous>:wasm-function[795]:0x3006d
 at <anonymous>:wasm-function[3628]:0xf01cc
[2023-05-24T23:29:51.006] [ERROR] Tdarr_Server - Error running MediaInfo 2
[2023-05-24T23:29:51.006] [ERROR] Tdarr_Server - RangeError: Maximum call stack size exceeded
 at x (<anonymous>:wasm-function[381]:0x15c4d)
 at <anonymous>:wasm-function[46]:0x5dc0
 at <anonymous>:wasm-function[652]:0x21cb9
 at <anonymous>:wasm-function[1023]:0x47018
 at <anonymous>:wasm-function[853]:0x37827
 at <anonymous>:wasm-function[3684]:0xf4884
 at <anonymous>:wasm-function[3516]:0xeb5b7
 at <anonymous>:wasm-function[1061]:0x487c9
 at <anonymous>:wasm-function[795]:0x3006d
 at <anonymous>:wasm-function[3628]:0xf01cc
[2023-05-24T23:29:58.220] [ERROR] Tdarr_Server - Error running MediaInfo 1
[2023-05-24T23:29:58.223] [ERROR] Tdarr_Server - RangeError: Maximum call stack size exceeded
 at x (<anonymous>:wasm-function[381]:0x15c4d)
 at <anonymous>:wasm-function[46]:0x5dc0
 at <anonymous>:wasm-function[652]:0x21cb9
 at <anonymous>:wasm-function[1023]:0x47018
 at <anonymous>:wasm-function[853]:0x37827
 at <anonymous>:wasm-function[3684]:0xf4884
 at <anonymous>:wasm-function[3516]:0xeb5b7
 at <anonymous>:wasm-function[1061]:0x487c9
 at <anonymous>:wasm-function[795]:0x3006d
 at <anonymous>:wasm-function[3628]:0xf01cc
[2023-05-24T23:29:58.224] [ERROR] Tdarr_Server - Error running MediaInfo 2
[2023-05-24T23:29:58.224] [ERROR] Tdarr_Server - RangeError: Maximum call stack size exceeded
 at x (<anonymous>:wasm-function[381]:0x15c4d)
 at <anonymous>:wasm-function[46]:0x5dc0
 at <anonymous>:wasm-function[652]:0x21cb9
 at <anonymous>:wasm-function[1023]:0x47018
 at <anonymous>:wasm-function[853]:0x37827
 at <anonymous>:wasm-function[3684]:0xf4884
 at <anonymous>:wasm-function[3516]:0xeb5b7
 at <anonymous>:wasm-function[1061]:0x487c9
 at <anonymous>:wasm-function[795]:0x3006d
 at <anonymous>:wasm-function[3628]:0xf01cc
</anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous></anonymous>

-
Failed to decode h264 key frame with DXVA2.0 because returned buffer is to small
16 mai 2023, par grill2010I hava a strange problem on Windows with DXVA2 h264 decoding. I recently figured out a ffmpeg decoding limitation for DXVA2 and D3D11VA on Windows and how to solve it, this solution completly fixes the problem with D3D11VA but DXVA2 still has some problems with certain keyframes. Upon further investigation it turned out that the decoding of these certain keyframes fail because the buffer returned from the IDirectXVideoDecoder_GetBuffer function was too small. FFmpeg is printing out these logs when the decoding fails :


Error: [h264 @ 0000028b2e5796c0] Buffer for type 5 was too small. size: 58752, dxva_size: 55296
Error: [h264 @ 0000028b2e5796c0] Failed to add bitstream or slice control buffer
Error: [h264 @ 0000028b2e5796c0] hardware accelerator failed to decode picture


Why is this returned buffer too low ? What kind of factors inside ffmpeg do have an effect on this buffer size or is this a limitation of DXVA2 in general ? All other decoders like Cuvid, D3D11VA or the software decoder are not affected by this problem and can decode all keyframes.


I have an example javacv project on github that can reproduce the problem. I also provide the source code of the main class here. The keyframe example data with prepended SPS and PPS in hex form can be downloaded here.


import javafx.application.Application;
import javafx.scene.Scene;
import javafx.scene.layout.Pane;
import javafx.scene.layout.StackPane;
import javafx.stage.Stage;
import org.bytedeco.ffmpeg.avcodec.AVCodec;
import org.bytedeco.ffmpeg.avcodec.AVCodecContext;
import org.bytedeco.ffmpeg.avcodec.AVCodecHWConfig;
import org.bytedeco.ffmpeg.avcodec.AVPacket;
import org.bytedeco.ffmpeg.avutil.AVBufferRef;
import org.bytedeco.ffmpeg.avutil.AVDictionary;
import org.bytedeco.ffmpeg.avutil.AVFrame;
import org.bytedeco.ffmpeg.avutil.LogCallback;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.IntPointer;
import org.bytedeco.javacpp.Pointer;
import org.tinylog.Logger;

import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.util.Objects;
import java.util.function.Consumer;

import static org.bytedeco.ffmpeg.avcodec.AVCodecContext.FF_THREAD_SLICE;
import static org.bytedeco.ffmpeg.global.avcodec.*;
import static org.bytedeco.ffmpeg.global.avutil.*;

public class App extends Application {

 /**** decoder variables ****/

 private AVHWContextInfo hardwareContext;

 private AVCodec decoder;
 private AVCodecContext m_VideoDecoderCtx;

 private AVCodecContext.Get_format_AVCodecContext_IntPointer formatCallback;

 private final int streamResolutionX = 1920;
 private final int streamResolutionY = 1080;

 // AV_HWDEVICE_TYPE_CUDA // example works with cuda
 // AV_HWDEVICE_TYPE_DXVA2 // producing Invalid data found on keyframe
 // AV_HWDEVICE_TYPE_D3D11VA // producing Invalid data found on keyframe
 private static final int HW_DEVICE_TYPE = AV_HWDEVICE_TYPE_DXVA2;

 private static final boolean USE_HW_ACCEL = true;

 private static final boolean USE_AV_EF_EXPLODE = true;

 public static void main(final String[] args) {
 //System.setProperty("prism.order", "d3d,sw");
 System.setProperty("prism.vsync", "false");
 Application.launch(App.class);
 }

 @Override
 public void start(final Stage primaryStage) {
 final Pane dummyPane = new Pane();
 dummyPane.setStyle("-fx-background-color: black");
 final Scene scene = new Scene(dummyPane, this.streamResolutionX, this.streamResolutionY);
 primaryStage.setScene(scene);
 primaryStage.show();
 primaryStage.setMinWidth(480);
 primaryStage.setMinHeight(360);

 this.initializeFFmpeg(result -> {
 if (!result) {
 Logger.error("FFmpeg could not be initialized correctly, terminating program");
 System.exit(1);
 return;
 }
 scene.setRoot(new StackPane());
 this.performTestFramesFeeding();
 });
 }

 private void initializeFFmpeg(final Consumer<boolean> finishHandler) {
 FFmpegLogCallback.setLevel(AV_LOG_DEBUG); // Increase log level until the first frame is decoded
 FFmpegLogCallback.set();
 Pointer pointer = new Pointer((Pointer) null);
 AVCodec c;
 while ((c = av_codec_iterate(pointer)) != null) {
 if (av_codec_is_decoder(c) > 0)
 Logger.debug("{}:{} ", c.name().getString(), c.type());
 }

 this.decoder = avcodec_find_decoder(AV_CODEC_ID_H264); // usually decoder name is h264 and without hardware support it's yuv420p otherwise nv12
 if (this.decoder == null) {
 Logger.error("Unable to find decoder for format {}", "h264");
 finishHandler.accept(false);
 return;
 }
 Logger.info("Current decoder name: {}, {}", this.decoder.name().getString(), this.decoder.long_name().getString());

 if (true) {
 for (; ; ) {
 this.m_VideoDecoderCtx = avcodec_alloc_context3(this.decoder);
 if (this.m_VideoDecoderCtx == null) {
 Logger.error("Unable to find decoder for format AV_CODEC_ID_H264");
 if (this.hardwareContext != null) {
 this.hardwareContext.free();
 this.hardwareContext = null;
 }
 continue;
 }

 if (App.USE_HW_ACCEL) {
 this.hardwareContext = this.createHardwareContext();
 if (this.hardwareContext != null) {
 Logger.info("Set hwaccel support");
 this.m_VideoDecoderCtx.hw_device_ctx(this.hardwareContext.hwContext()); // comment to disable hwaccel
 }
 } else {
 Logger.info("Hwaccel manually disabled");
 }

 // Always request low delay decoding
 this.m_VideoDecoderCtx.flags(this.m_VideoDecoderCtx.flags() | AV_CODEC_FLAG_LOW_DELAY);

 // Allow display of corrupt frames and frames missing references
 this.m_VideoDecoderCtx.flags(this.m_VideoDecoderCtx.flags() | AV_CODEC_FLAG_OUTPUT_CORRUPT);
 this.m_VideoDecoderCtx.flags2(this.m_VideoDecoderCtx.flags2() | AV_CODEC_FLAG2_SHOW_ALL);

 if (App.USE_AV_EF_EXPLODE) {
 // Report decoding errors to allow us to request a key frame
 this.m_VideoDecoderCtx.err_recognition(this.m_VideoDecoderCtx.err_recognition() | AV_EF_EXPLODE);
 }

 // Enable slice multi-threading for software decoding
 if (this.m_VideoDecoderCtx.hw_device_ctx() == null) { // if not hw accelerated
 this.m_VideoDecoderCtx.thread_type(this.m_VideoDecoderCtx.thread_type() | FF_THREAD_SLICE);
 this.m_VideoDecoderCtx.thread_count(2/*AppUtil.getCpuCount()*/);
 } else {
 // No threading for HW decode
 this.m_VideoDecoderCtx.thread_count(1);
 }

 this.m_VideoDecoderCtx.width(this.streamResolutionX);
 this.m_VideoDecoderCtx.height(this.streamResolutionY);
 this.m_VideoDecoderCtx.pix_fmt(this.getDefaultPixelFormat());

 this.formatCallback = new AVCodecContext.Get_format_AVCodecContext_IntPointer() {
 @Override
 public int call(final AVCodecContext context, final IntPointer pixelFormats) {
 final boolean hwDecodingSupported = context.hw_device_ctx() != null && App.this.hardwareContext != null;
 final int preferredPixelFormat = hwDecodingSupported ?
 App.this.hardwareContext.hwConfig().pix_fmt() :
 context.pix_fmt();
 int i = 0;
 while (true) {
 final int currentSupportedFormat = pixelFormats.get(i++);
 System.out.println("Supported pixel formats " + currentSupportedFormat);
 if (currentSupportedFormat == AV_PIX_FMT_NONE) {
 break;
 }
 }

 i = 0;
 while (true) {
 final int currentSupportedFormat = pixelFormats.get(i++);
 if (currentSupportedFormat == preferredPixelFormat) {
 Logger.info("[FFmpeg]: pixel format in format callback is {}", currentSupportedFormat);
 return currentSupportedFormat;
 }
 if (currentSupportedFormat == AV_PIX_FMT_NONE) {
 break;
 }
 }

 i = 0;
 while (true) { // try again and search for yuv
 final int currentSupportedFormat = pixelFormats.get(i++);
 if (currentSupportedFormat == AV_PIX_FMT_YUV420P) {
 Logger.info("[FFmpeg]: Not found in first match so use {}", AV_PIX_FMT_YUV420P);
 return currentSupportedFormat;
 }
 if (currentSupportedFormat == AV_PIX_FMT_NONE) {
 break;
 }
 }

 i = 0;
 while (true) { // try again and search for nv12
 final int currentSupportedFormat = pixelFormats.get(i++);
 if (currentSupportedFormat == AV_PIX_FMT_NV12) {
 Logger.info("[FFmpeg]: Not found in second match so use {}", AV_PIX_FMT_NV12);
 return currentSupportedFormat;
 }
 if (currentSupportedFormat == AV_PIX_FMT_NONE) {
 break;
 }
 }

 Logger.info("[FFmpeg]: pixel format in format callback is using fallback {}", AV_PIX_FMT_NONE);
 return AV_PIX_FMT_NONE;
 }
 };
 this.m_VideoDecoderCtx.get_format(this.formatCallback);

 final AVDictionary options = new AVDictionary(null);
 final int result = avcodec_open2(this.m_VideoDecoderCtx, this.decoder, options);
 if (result < 0) {
 Logger.error("avcodec_open2 was not successful");
 finishHandler.accept(false);
 return;
 }
 av_dict_free(options);
 break;
 }
 }

 if (this.decoder == null || this.m_VideoDecoderCtx == null) {
 finishHandler.accept(false);
 return;
 }
 finishHandler.accept(true);
 }

 private AVHWContextInfo createHardwareContext() {
 AVHWContextInfo result = null;
 for (int i = 0; ; i++) {
 final AVCodecHWConfig config = avcodec_get_hw_config(this.decoder, i);
 if (config == null) {
 break;
 }

 if ((config.methods() & AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX) < 0) {
 continue;
 }
 final int device_type = config.device_type();
 if (device_type != App.HW_DEVICE_TYPE) {
 continue;
 }
 final AVBufferRef hw_context = av_hwdevice_ctx_alloc(device_type);
 if (hw_context == null || av_hwdevice_ctx_create(hw_context, device_type, (String) null, null, 0) < 0) {
 Logger.error("HW accel not supported for type {}", device_type);
 av_free(config);
 av_free(hw_context);
 } else {
 Logger.info("HW accel created for type {}", device_type);
 result = new AVHWContextInfo(config, hw_context);
 }
 break;
 }

 return result;
 }

 @Override
 public void stop() {
 this.releaseNativeResources();
 }

 /*****************************/
 /*** test frame processing ***/
 /*****************************/
 
 private void performTestFramesFeeding() {
 final AVPacket pkt = av_packet_alloc();
 if (pkt == null) {
 return;
 }
 try (final BytePointer bp = new BytePointer(65_535 * 15)) {


 for (int i = 0; i < 1; i++) {
 final byte[] frameData = AVTestFrames.h264KeyTestFrame;

 bp.position(0);

 bp.put(frameData);
 bp.limit(frameData.length);

 pkt.data(bp);
 pkt.capacity(bp.capacity());
 pkt.size(frameData.length);
 pkt.position(0);
 pkt.limit(frameData.length);
 //pkt.flags(AV_PKT_FLAG_KEY);
 final AVFrame avFrame = av_frame_alloc();
 System.out.println("frameData.length " + frameData.length);

 final int err = avcodec_send_packet(this.m_VideoDecoderCtx, pkt); //fill_scaling_lists
 if (err < 0) {
 final BytePointer buffer = new BytePointer(512);
 av_strerror(err, buffer, buffer.capacity());
 final String string = buffer.getString();
 System.out.println("Error on decoding test frame " + err + " message " + string);
 av_frame_free(avFrame);
 return;
 }

 final int result = avcodec_receive_frame(this.m_VideoDecoderCtx, avFrame);
 final AVFrame decodedFrame;
 if (result == 0) {
 if (this.m_VideoDecoderCtx.hw_device_ctx() == null) {
 decodedFrame = avFrame;
 System.out.println("SUCESS with SW decoding");
 } else {
 final AVFrame hwAvFrame = av_frame_alloc();
 if (av_hwframe_transfer_data(hwAvFrame, avFrame, 0) < 0) {
 System.out.println("Failed to transfer frame from hardware");
 av_frame_unref(hwAvFrame);
 decodedFrame = avFrame;
 } else {
 av_frame_unref(avFrame);
 decodedFrame = hwAvFrame;
 System.out.println("SUCESS with HW decoding");
 }
 }

 av_frame_unref(decodedFrame);
 } else {
 final BytePointer buffer = new BytePointer(512);
 av_strerror(result, buffer, buffer.capacity());
 final String string = buffer.getString();
 System.out.println("error " + result + " message " + string);
 av_frame_free(avFrame);
 }
 }
 } finally {
 if (pkt.stream_index() != -1) {
 av_packet_unref(pkt);
 }
 pkt.releaseReference();
 }
 }

 final Object releaseLock = new Object();
 private volatile boolean released = false;

 private void releaseNativeResources() {
 if (this.released) {
 return;
 }
 this.released = true;
 synchronized (this.releaseLock) {
 // Close the video codec
 if (this.m_VideoDecoderCtx != null) {
 avcodec_free_context(this.m_VideoDecoderCtx);
 this.m_VideoDecoderCtx = null;
 }

 // close the format callback
 if (this.formatCallback != null) {
 this.formatCallback.close();
 this.formatCallback = null;
 }

 // close hw context
 if (this.hardwareContext != null) {
 this.hardwareContext.free();
 }
 }
 }

 private int getDefaultPixelFormat() {
 return AV_PIX_FMT_YUV420P; // Always return yuv420p here
 }


 /*********************/
 /*** inner classes ***/
 /*********************/

 public static final class HexUtil {

 private HexUtil() {
 }

 public static byte[] unhexlify(final String argbuf) {
 final int arglen = argbuf.length();
 if (arglen % 2 != 0) {
 throw new RuntimeException("Odd-length string");
 } else {
 final byte[] retbuf = new byte[arglen / 2];

 for (int i = 0; i < arglen; i += 2) {
 final int top = Character.digit(argbuf.charAt(i), 16);
 final int bot = Character.digit(argbuf.charAt(i + 1), 16);
 if (top == -1 || bot == -1) {
 throw new RuntimeException("Non-hexadecimal digit found");
 }

 retbuf[i / 2] = (byte) ((top << 4) + bot);
 }

 return retbuf;
 }
 }
 }

 public static final class AVHWContextInfo {
 private final AVCodecHWConfig hwConfig;
 private final AVBufferRef hwContext;

 private volatile boolean freed = false;

 public AVHWContextInfo(final AVCodecHWConfig hwConfig, final AVBufferRef hwContext) {
 this.hwConfig = hwConfig;
 this.hwContext = hwContext;
 }

 public AVCodecHWConfig hwConfig() {
 return this.hwConfig;
 }

 public AVBufferRef hwContext() {
 return this.hwContext;
 }

 public void free() {
 if (this.freed) {
 return;
 }
 this.freed = true;
 av_free(this.hwConfig);
 av_free(this.hwContext);
 }


 @Override
 public boolean equals(Object o) {
 if (this == o) return true;
 if (o == null || getClass() != o.getClass()) return false;
 AVHWContextInfo that = (AVHWContextInfo) o;
 return freed == that.freed && Objects.equals(hwConfig, that.hwConfig) && Objects.equals(hwContext, that.hwContext);
 }

 @Override
 public int hashCode() {
 return Objects.hash(hwConfig, hwContext, freed);
 }

 @Override
 public String toString() {
 return "AVHWContextInfo[" +
 "hwConfig=" + this.hwConfig + ", " +
 "hwContext=" + this.hwContext + ']';
 }
 }

 public static final class AVTestFrames {

 private AVTestFrames() {

 }

 static {
 InputStream inputStream = null;
 try {
 inputStream = AVTestFrames.class.getClassLoader().getResourceAsStream("h264_test_key_frame.txt");
 final byte[] h264TestFrameBuffer = inputStream == null ? new byte[0] : inputStream.readAllBytes();
 final String h264TestFrame = new String(h264TestFrameBuffer, StandardCharsets.UTF_8);
 AVTestFrames.h264KeyTestFrame = HexUtil.unhexlify(h264TestFrame);
 } catch (final IOException e) {
 Logger.error(e, "Could not parse test frame");
 } finally {
 if (inputStream != null) {
 try {
 inputStream.close();
 } catch (final IOException e) {
 Logger.error(e, "Could not close test frame input stream");
 }
 }
 }
 }

 public static byte[] h264KeyTestFrame;
 }

 public static class FFmpegLogCallback extends LogCallback {

 private static final org.bytedeco.javacpp.tools.Logger logger = org.bytedeco.javacpp.tools.Logger.create(FFmpegLogCallback.class);

 static final FFmpegLogCallback instance = new FFmpegLogCallback().retainReference();

 public static FFmpegLogCallback getInstance() {
 return instance;
 }

 /**
 * Calls {@code avutil.setLogCallback(getInstance())}.
 */
 public static void set() {
 setLogCallback(getInstance());
 }

 /**
 * Returns {@code av_log_get_level()}.
 **/
 public static int getLevel() {
 return av_log_get_level();
 }

 /**
 * Calls {@code av_log_set_level(level)}.
 **/
 public static void setLevel(int level) {
 av_log_set_level(level);
 }

 @Override
 public void call(int level, BytePointer msg) {
 switch (level) {
 case AV_LOG_PANIC, AV_LOG_FATAL, AV_LOG_ERROR -> logger.error(msg.getString());
 case AV_LOG_WARNING -> logger.warn(msg.getString());
 case AV_LOG_INFO -> logger.info(msg.getString());
 case AV_LOG_VERBOSE, AV_LOG_DEBUG, AV_LOG_TRACE -> logger.debug(msg.getString());
 default -> {
 assert false;
 }
 }
 }
 }
}
</boolean>

-
Downscaling a video from 1080p to 480p using swscale and encoding to x265 gives a glitched output
5 mai 2023, par lokit khemkaI am basically first scaling a frame and then sending the frame to the encoder as below :


scaled_frame->pts = input_frame->pts;
scaled_frame->pkt_dts = input_frame->pkt_dts;
scaled_frame->pict_type = input_frame->pict_type;
sws_scale_frame(encoder->sws_ctx, scaled_frame, input_frame);
if (encode_video(decoder, encoder, scaled_frame))
 return -1;


The scaling context is configured as :


scaled_frame->width = 854;
scaled_frame->height=480; 
encoder->sws_ctx = sws_getContext(1920, 1080,
 decoder->video_avcc->pix_fmt, 
 scaled_frame->width, scaled_frame->height, decoder->video_avcc->pix_fmt, SWS_BICUBIC, NULL, NULL, NULL );
 if (!encoder->sws_ctx){logging("Cannot Create Scaling Context."); return -1;}


The encoder is configured as :


encoder_sc->video_avcc->height = decoder_ctx->height; //1080
 encoder_sc->video_avcc->width = decoder_ctx->width; //1920
 encoder_sc->video_avcc->bit_rate = 2 * 1000 * 1000;
 encoder_sc->video_avcc->rc_buffer_size = 4 * 1000 * 1000;
 encoder_sc->video_avcc->rc_max_rate = 2 * 1000 * 1000;
 encoder_sc->video_avcc->rc_min_rate = 2.5 * 1000 * 1000;

 encoder_sc->video_avcc->time_base = av_inv_q(input_framerate);
 encoder_sc->video_avs->time_base = encoder_sc->video_avcc->time_base;


When I get the output, the output video is 1080p and I have glitches like :


I changed the encoder avcc resolution to 480p (854 x 480). However, that is causing the video to get sliced to the top quarter of the original frame.
I am new to FFMPEG and video processing in general.


EDIT : I am adding the minimal reproducible code sample. However, it is really long because I need to include code for decoding, scaling and then encoding because the possible error is either in scaling or encoding :


#include <libavcodec></libavcodec>avcodec.h>
#include <libavformat></libavformat>avformat.h>
#include <libavutil></libavutil>timestamp.h>
#include <libavutil></libavutil>opt.h>
#include <libswscale></libswscale>swscale.h>

#include 
#include 

typedef struct StreamingContext{
 AVFormatContext* avfc;
 AVCodec *video_avc;
 AVCodec *audio_avc;
 AVStream *video_avs;
 AVStream *audio_avs;
 AVCodecContext *video_avcc;
 AVCodecContext *audio_avcc;
 int video_index;
 int audio_index;
 char* filename;
 struct SwsContext *sws_ctx;
}StreamingContext;


typedef struct StreamingParams{
 char copy_video;
 char copy_audio;
 char *output_extension;
 char *muxer_opt_key;
 char *muxer_opt_value;
 char *video_codec;
 char *audio_codec;
 char *codec_priv_key;
 char *codec_priv_value;
}StreamingParams;

void logging(const char *fmt, ...)
{
 va_list args;
 fprintf(stderr, "LOG: ");
 va_start(args, fmt);
 vfprintf(stderr, fmt, args);
 va_end(args);
 fprintf(stderr, "\n");
}

int fill_stream_info(AVStream *avs, AVCodec **avc, AVCodecContext **avcc)
{
 *avc = avcodec_find_decoder(avs->codecpar->codec_id);
 if (!*avc)
 {
 logging("Failed to find the codec.\n");
 return -1;
 }

 *avcc = avcodec_alloc_context3(*avc);
 if (!*avcc)
 {
 logging("Failed to alloc memory for codec context.");
 return -1;
 }

 if (avcodec_parameters_to_context(*avcc, avs->codecpar) < 0)
 {
 logging("Failed to fill Codec Context.");
 return -1;
 }

 if (avcodec_open2(*avcc, *avc, NULL) < 0)
 {
 logging("Failed to open Codec.");
 return -1;
 }

 return 0;
}

int open_media(const char *in_filename, AVFormatContext **avfc)
{
 *avfc = avformat_alloc_context();

 if (!*avfc)
 {
 logging("Failed to Allocate Memory for Format Context");
 return -1;
 }

 if (avformat_open_input(avfc, in_filename, NULL, NULL) != 0)
 {
 logging("Failed to open input file %s", in_filename);
 return -1;
 }

 if (avformat_find_stream_info(*avfc, NULL) < 0)
 {
 logging("Failed to get Stream Info.");
 return -1;
 }
}

int prepare_decoder(StreamingContext *sc)
{
 for (int i = 0; i < sc->avfc->nb_streams; i++)
 {
 if (sc->avfc->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO)
 {
 sc->video_avs = sc->avfc->streams[i];
 sc->video_index = i;

 if (fill_stream_info(sc->video_avs, &sc->video_avc, &sc->video_avcc))
 {
 return -1;
 }
 }
 else
 {
 logging("Skipping Streams other than Video.");
 }
 }
 return 0;
}

int prepare_video_encoder(StreamingContext *encoder_sc, AVCodecContext *decoder_ctx, AVRational input_framerate,
 StreamingParams sp)
{
 encoder_sc->video_avs = avformat_new_stream(encoder_sc->avfc, NULL);
 encoder_sc->video_avc = avcodec_find_encoder_by_name(sp.video_codec);
 if (!encoder_sc->video_avc)
 {
 logging("Cannot find the Codec.");
 return -1;
 }

 encoder_sc->video_avcc = avcodec_alloc_context3(encoder_sc->video_avc);
 if (!encoder_sc->video_avcc)
 {
 logging("Could not allocate memory for Codec Context.");
 return -1;
 }

 av_opt_set(encoder_sc->video_avcc->priv_data, "preset", "fast", 0);
 if (sp.codec_priv_key && sp.codec_priv_value)
 av_opt_set(encoder_sc->video_avcc->priv_data, sp.codec_priv_key, sp.codec_priv_value, 0);

 encoder_sc->video_avcc->height = decoder_ctx->height;
 encoder_sc->video_avcc->width = decoder_ctx->width;
 encoder_sc->video_avcc->sample_aspect_ratio = decoder_ctx->sample_aspect_ratio;

 if (encoder_sc->video_avc->pix_fmts)
 encoder_sc->video_avcc->pix_fmt = encoder_sc->video_avc->pix_fmts[0];
 else
 encoder_sc->video_avcc->pix_fmt = decoder_ctx->pix_fmt;

 encoder_sc->video_avcc->bit_rate = 2 * 1000 * 1000;
 encoder_sc->video_avcc->rc_buffer_size = 4 * 1000 * 1000;
 encoder_sc->video_avcc->rc_max_rate = 2 * 1000 * 1000;
 encoder_sc->video_avcc->rc_min_rate = 2.5 * 1000 * 1000;

 encoder_sc->video_avcc->time_base = av_inv_q(input_framerate);
 encoder_sc->video_avs->time_base = encoder_sc->video_avcc->time_base;

 

 if (avcodec_open2(encoder_sc->video_avcc, encoder_sc->video_avc, NULL) < 0)
 {
 logging("Could not open the Codec.");
 return -1;
 }
 avcodec_parameters_from_context(encoder_sc->video_avs->codecpar, encoder_sc->video_avcc);
 return 0;
}

int encode_video(StreamingContext *decoder, StreamingContext *encoder, AVFrame *input_frame)
{
 if (input_frame)
 input_frame->pict_type = AV_PICTURE_TYPE_NONE;

 AVPacket *output_packet = av_packet_alloc();
 if (!output_packet)
 {
 logging("Could not allocate memory for Output Packet.");
 return -1;
 }

 int response = avcodec_send_frame(encoder->video_avcc, input_frame);

 while (response >= 0)
 {
 response = avcodec_receive_packet(encoder->video_avcc, output_packet);
 if (response == AVERROR(EAGAIN) || response == AVERROR_EOF)
 {
 break;
 }
 else if (response < 0)
 {
 logging("Error while receiving packet from encoder: %s", av_err2str(response));
 return -1;
 }

 output_packet->stream_index = decoder->video_index;
 output_packet->duration = encoder->video_avs->time_base.den / encoder->video_avs->time_base.num / decoder->video_avs->avg_frame_rate.num * decoder->video_avs->avg_frame_rate.den;

 av_packet_rescale_ts(output_packet, decoder->video_avs->time_base, encoder->video_avs->time_base);
 response = av_interleaved_write_frame(encoder->avfc, output_packet);
 if (response != 0)
 {
 logging("Error %d while receiving packet from decoder: %s", response, av_err2str(response));
 return -1;
 }
 }

 av_packet_unref(output_packet);
 av_packet_free(&output_packet);

 return 0;
}

int transcode_video(StreamingContext *decoder, StreamingContext *encoder, AVPacket *input_packet, AVFrame *input_frame, AVFrame *scaled_frame)
{
 int response = avcodec_send_packet(decoder->video_avcc, input_packet);
 if (response < 0)
 {
 logging("Error while sending the Packet to Decoder: %s", av_err2str(response));
 return response;
 }

 while (response >= 0)
 {
 response = avcodec_receive_frame(decoder->video_avcc, input_frame);
 
 if (response == AVERROR(EAGAIN) || response == AVERROR_EOF)
 {
 break;
 }
 else if (response < 0)
 {
 logging("Error while receiving frame from Decoder: %s", av_err2str(response));
 return response;
 }
 if (response >= 0)
 {
 scaled_frame->pts = input_frame->pts;
 scaled_frame->pkt_dts = input_frame->pkt_dts;
 scaled_frame->pict_type = input_frame->pict_type;
 sws_scale_frame(encoder->sws_ctx, scaled_frame, input_frame);
 if (encode_video(decoder, encoder, scaled_frame))
 return -1;
 }

 av_frame_unref(input_frame);
 }
 return 0;
}

int main(int argc, char *argv[])
{
 StreamingParams sp = {0};
 sp.copy_audio = 1;
 sp.copy_video = 0;
 sp.video_codec = "libx265";


 StreamingContext *decoder = (StreamingContext *)calloc(1, sizeof(StreamingContext));
 decoder->filename = argv[1];

 StreamingContext *encoder = (StreamingContext *)calloc(1, sizeof(StreamingContext));
 encoder->filename = argv[2];

 if (sp.output_extension)
 {
 strcat(encoder->filename, sp.output_extension);
 }

 if (open_media(decoder->filename, &decoder->avfc))
 return -1;
 if (prepare_decoder(decoder))
 return -1;

 avformat_alloc_output_context2(&encoder->avfc, NULL, NULL, encoder->filename);
 if (!encoder->avfc)
 {
 logging("Could not allocate memory for output Format Context.");
 return -1;
 }

 AVRational input_framerate = av_guess_frame_rate(decoder->avfc, decoder->video_avs, NULL);
 prepare_video_encoder(encoder, decoder->video_avcc, input_framerate, sp);


 if (encoder->avfc->oformat->flags & AVFMT_GLOBALHEADER)
 encoder->avfc->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;

 if (!(encoder->avfc->oformat->flags & AVFMT_NOFILE))
 {
 if (avio_open(&encoder->avfc->pb, encoder->filename, AVIO_FLAG_WRITE) < 0)
 {
 logging("could not open the output file");
 return -1;
 }
 }

 AVDictionary *muxer_opts = NULL;

 if (sp.muxer_opt_key && sp.muxer_opt_value)
 {
 av_dict_set(&muxer_opts, sp.muxer_opt_key, sp.muxer_opt_value, 0);
 }

 if (avformat_write_header(encoder->avfc, &muxer_opts) < 0)
 {
 logging("an error occurred when opening output file");
 return -1;
 }

 AVFrame *input_frame = av_frame_alloc();
 AVFrame *scaled_frame = av_frame_alloc();
 if (!input_frame || !scaled_frame)
 {
 logging("Failed to allocate memory for AVFrame");
 return -1;
 }

 // scaled_frame->format = AV_PIX_FMT_YUV420P;
 scaled_frame->width = 854;
 scaled_frame->height=480; 

 //Creating Scaling Context
 encoder->sws_ctx = sws_getContext(1920, 1080,
 decoder->video_avcc->pix_fmt, 
 scaled_frame->width, scaled_frame->height, decoder->video_avcc->pix_fmt, SWS_BICUBIC, NULL, NULL, NULL );
 if (!encoder->sws_ctx){logging("Cannot Create Scaling Context."); return -1;}


 AVPacket *input_packet = av_packet_alloc();
 if (!input_packet)
 {
 logging("Failed to allocate memory for AVPacket.");
 return -1;
 }

 while (av_read_frame(decoder->avfc, input_packet) >= 0)
 {
 if (decoder->avfc->streams[input_packet->stream_index]->codecpar->codec_type == AVMEDIA_TYPE_VIDEO)
 {
 if (transcode_video(decoder, encoder, input_packet, input_frame, scaled_frame))
 return -1;
 av_packet_unref(input_packet);
 }
 else
 {
 logging("Ignoring all nonvideo packets.");
 }
 }

 if (encode_video(decoder, encoder, NULL))
 return -1;

 av_write_trailer(encoder->avfc);

 if (muxer_opts != NULL)
 {
 av_dict_free(&muxer_opts);
 muxer_opts = NULL;
 }

 if (input_frame != NULL)
 {
 av_frame_free(&input_frame);
 input_frame = NULL;
 }

 if (input_packet != NULL)
 {
 av_packet_free(&input_packet);
 input_packet = NULL;
 }

 avformat_close_input(&decoder->avfc);

 avformat_free_context(decoder->avfc);
 decoder->avfc = NULL;
 avformat_free_context(encoder->avfc);
 encoder->avfc = NULL;

 avcodec_free_context(&decoder->video_avcc);
 decoder->video_avcc = NULL;
 avcodec_free_context(&decoder->audio_avcc);
 decoder->audio_avcc = NULL;

 free(decoder);
 decoder = NULL;
 free(encoder);
 encoder = NULL;

 return 0;
}


The video I am using for testing is available at the repo : https://github.com/leandromoreira/ffmpeg-libav-tutorial


The file name is small_bunny_1080p_60fps.mp4


Syndiquer tout le site
