Newest 'ffmpeg' Questions - Stack Overflow
Articles published on the website
-
Find frames in the video that are similar to the static image, to crop the video
13 September, by Nguyen Van KufuI use opencv with ffmpeg to cut the video. Specifically, find the frame in the video that is the same as the picture, then cut the video at the time the video frame is the same as the picture.
I found the code on google and edited it according to my needs, but when it comes to comparing frames and still images, I don't know how to ask you to add a comparison section to cut the video.
import numpy as np import cv2 import time from skimage.metrics import structural_similarity as compare_ssim hinh = cv2.imread("f:\\x.png") hinh1 = cv2.resize(hinh, (500, 300)) cap = cv2.VideoCapture('f:\\tt.mp4') prev_frame_time = 0 new_frame_time = 0 cv2.imshow('hinh', hinh1) while(cap.isOpened()): ret, frame = cap.read() if not ret: break gray = frame gray = cv2.resize(gray, (500, 300)) font = cv2.FONT_HERSHEY_SIMPLEX new_frame_time = time.time() fps = 1/(new_frame_time-prev_frame_time) prev_frame_time = new_frame_time fps = int(fps) fps = str(fps) cv2.putText(gray, fps, (7, 70), font, 3, (100, 255, 0), 3, cv2.LINE_AA) ############## # displaying the frame with fps cv2.imshow('frame', gray) # press 'Q' if you want to exit if cv2.waitKey(1) & 0xFF == ord('q'): break cap.release() cv2.destroyAllWindows() -
Can't find error in function for changing sampling rate [closed]
12 September, by leeeeeeeeessI have function for changing sampling rate of audio (only one channel):
int change_sampling_rate(float *audio_input, int input_sample_rate, int output_sample_rate, int input_num_of_samples, float **audio_output, int *result_num_of_samples)When I run tests on time lag between two files (mp3) with different sampling rates, it gives answer that differs on about 15-20 ms with right answer. Can anybody, please, help me find mistakes in the code?
For example, I have two audios: [audio_1] (https://jmp.sh/s/USFPaGnHXVuKFVYarYpm) and [audio_2] (https://jmp.sh/s/jbmWbPTwkdDujAocmi56) - second audio is just a sample of first. The answer should be
35264 ms, but my function gives35249 ms:( -
Streaming issues with HLS setup using Nginx and FFmpeg, and TS video files
12 September, by Jacob AndersonI've been working on setting up an HLS stream on my Raspberry Pi to broadcast video from a security camera that's physically connected to my Raspberry Pi through my web server, making it accessible via my website. The
.tsvideo files and the.m3u8playlist are correctly being served from/var/www/html/hls. However, when I attempt to load the stream on Safari (as well as other browsers), the video continuously appears to be loading without ever displaying any content.Here are some details about my setup:
- Camera: I am using an Arducam 1080p Day & Night Vision USB Camera which is available on
/dev/video0. - Server Configuration: I haven't noticed any errors in the Safari console or on the server logs. When I access the
.tsfiles directly from the browser, they only show a black screen but they do play.
Given the situation, I suspect there might be an issue with my FFmpeg command or possibly with my Nginx configuration.
Here is what I have:
ffmpeg stream service:
/etc/systemd/system/ffmpeg-stream.service[Unit] Description=FFmpeg RTMP Stream After=network.target [Service] ExecStart=/usr/local/bin/start_ffmpeg.sh Restart=always User=jacobanderson Group=jacobanderson StandardError=syslog SyslogIdentifier=ffmpeg-stream Environment=FFMPEG_LOGLEVEL=error [Install] WantedBy=multi-user.targetffmpeg command:
/usr/local/bin/start_ffmpeg.sh#!/bin/bash /usr/bin/ffmpeg -f v4l2 -input_format mjpeg -video_size 1280x720 -framerate 30 -i /dev/video0 -vcodec libx264 -preset veryfast -acodec aac -strict -2 -f flv rtmp://localhost/live/nginx.conf:
/etc/nginx/nginx.confuser www-data; worker_processes auto; pid /run/nginx.pid; error_log /var/log/nginx/error.log; include /etc/nginx/modules-enabled/*.conf; events { worker_connections 768; # multi_accept on; } rtmp { server { listen 1935; chunk_size 4096; #allow publish 127.0.0.1; #deny publish all; application live { #allow 192.168.0.100; live on; hls on; hls_path /var/www/html/hls; hls_fragment 3; hls_nested on; #hls_fragment_naming stream; hls_playlist_length 120; hls_cleanup on; hls_continuous on; #deny play all; } } } http { ## # Basic Settings ## sendfile on; #sendfile off; tcp_nopush on; types_hash_max_size 2048; # server_tokens off; # Additional for video directio 512; # server_names_hash_bucket_size 64; # server_name_in_redirect off; include /etc/nginx/mime.types; default_type application/octet-stream; ## # SSL Settings ## ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; #ssl_protocols TLSv1.2 TLSv1.3; # Use only secure protocols ssl_prefer_server_ciphers on; #ssl_ciphers "HIGH:!aNULL:!MD5"; ## # Logging Settings ## access_log /var/log/nginx/access.log; error_log /var/log/nginx/error.log; ## # Gzip Settings ## #gzip on; gzip off; # Ensure gzip is off for HLS ## # Virtual Host Configs ## include /etc/nginx/conf.d/*.conf; include /etc/nginx/sites-enabled/*; }sites-available:
/etc/nginx/sites-available/myStream.mysite.comserver { listen 443 ssl; server_name myStream.mysite.com; ssl_certificate /etc/letsencrypt/live/myStream.mysite.com/fullchain.pem; # managed by Certbot ssl_certificate_key /etc/letsencrypt/live/myStream.mysite.com/privkey.pem; # managed by Certbot include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot location / { root /var/www/html/hls; index index.html; } location /hls { # Password protection auth_basic "Restricted Content"; auth_basic_user_file /etc/nginx/.htpasswd; # Disable cache add_header Cache-Control no-cache; # CORS setup add_header 'Access-Control-Allow-Origin' '*' always; add_header 'Access-Control-Expose-Headers' 'Content-Length'; # Allow CORS preflight requests if ($request_method = 'OPTIONS') { add_header 'Access-Control-Allow-Origin' '*'; add_header 'Access-Control-Max-Age' 1728000; add_header 'Content-Type' 'text/plain charset=UTF-8'; add_header 'Content-Length' 0; return 204; } types { application/vnd.apple.mpegurl m3u8; video/mp2t ts; text/html html; text/css css; } root /var/www/html; } } server { listen 80; server_name myStream.mysite.com; if ($host = myStream.mysite.com) { return 301 https://$host$request_uri; } return 404; # managed by Certbot }index.html:
/var/www/html/hls/index.html<script src='http://stackoverflow.com/feeds/tag/js/hls.min.js'></script><script src="https://vjs.zencdn.net/7.10.2/video.js"></script><script>
 if (Hls.isSupported()) {
 var video = document.getElementById('my-video_html5_api'); // Updated ID to target the correct video element
 var hls = new Hls();
 hls.loadSource('https://myStream.mysite.com/hls/index.m3u8');
 hls.attachMedia(video);
 hls.on(Hls.Events.MANIFEST_PARSED,function() {
 video.play();
 });
 } else if (video.canPlayType('application/vnd.apple.mpegurl')) {
 video.src = 'https://myStream.mysite.com/hls/index.m3u8';
 video.addEventListener('loadedmetadata', function() {
 video.play();
 });
 }
 </script>Has anyone experienced similar issues or can spot an error in my configuration? Any help would be greatly appreciated as I have already invested over 30 hours trying to resolve this.
- Camera: I am using an Arducam 1080p Day & Night Vision USB Camera which is available on
-
SPM binary target, additional frameworks not being included
12 September, by DarrenI'm trying to integrate a
binaryTargetinto a project usingSwift Package Manager. The binary comes as a.zipcontaining the mainffmpegkit.xcframeworkplus 7 additional.xcframework's that it depends on.Here's the folder structure from
DerivedData/.../SourcePackages/artifacts/...afterSPMhas unzipped it.
This is my
Package.swiftfile:// swift-tools-version: 5.10 import PackageDescription let package = Package( name: "converter", platforms: [ .macOS(.v13), .iOS(.v14), ], products: [ .library( name: "converter", targets: ["converter"]) ], targets: [ .target( name: "converter", dependencies: [ .target(name: "ffmpeg-iOS", condition: .when(platforms: [.iOS])), .target(name: "ffmpeg-macOS", condition: .when(platforms: [.macOS])) ], path: "Sources/converter" ), .binaryTarget(name: "ffmpeg-iOS", url: "https://github.com/arthenica/ffmpeg-kit/releases/download/v6.0/ffmpeg-kit-full-6.0-ios-xcframework.zip", checksum: "c87ea1c77f0a8a6ba396c22fc33e9321befb8e85f8e8103046ddeb74fea66182"), .binaryTarget(name: "ffmpeg-macOS", url: "https://github.com/arthenica/ffmpeg-kit/releases/download/v6.0/ffmpeg-kit-full-6.0-macos-xcframework.zip", checksum: "8cab26eecd43b9389d37f64efaf43b9c6baf4e53614b62e6209d8ee8681b94b9") ] )Now when I build and run the project, it crashes with
dyld[85157]: Library not loaded: @rpatherrors as the build only seems to include theffmpegkit.xcframeworkfolder and not the others.This happens even if I do not
import ffmpegkitin any of my code. So what is making the build choose to add theffmpegkitframework but not the others?How can I tell SPM to include these additional frameworks when it builds?
-
Convert raw PCM data to OGG format using FFmpeg?
12 September, by Chris PAudioSegment AudioSegment::from_file(const std::string& file_path, const std::string& format, const std::string& codec, const std::map& parameters, int start_second, int duration) { avformat_network_init(); av_log_set_level(AV_LOG_ERROR); AVFormatContext* format_ctx = nullptr; if (avformat_open_input(&format_ctx, file_path.c_str(), nullptr, nullptr) != 0) { std::cerr << "Error: Could not open audio file." << std::endl; return AudioSegment(); } if (avformat_find_stream_info(format_ctx, nullptr) < 0) { std::cerr << "Error: Could not find stream information." << std::endl; avformat_close_input(&format_ctx); return AudioSegment(); } int audio_stream_index = -1; for (unsigned int i = 0; i < format_ctx->nb_streams; i++) { if (format_ctx->streams[i]->codecpar->codec_type == AVMEDIA_TYPE_AUDIO) { audio_stream_index = i; break; } } if (audio_stream_index == -1) { std::cerr << "Error: Could not find audio stream." << std::endl; avformat_close_input(&format_ctx); return AudioSegment(); } AVCodecParameters* codec_par = format_ctx->streams[audio_stream_index]->codecpar; const AVCodec* my_codec = avcodec_find_decoder(codec_par->codec_id); AVCodecContext* codec_ctx = avcodec_alloc_context3(my_codec); if (!codec_ctx) { std::cerr << "Error: Could not allocate codec context." << std::endl; avformat_close_input(&format_ctx); return AudioSegment(); } if (avcodec_parameters_to_context(codec_ctx, codec_par) < 0) { std::cerr << "Error: Could not initialize codec context." << std::endl; avcodec_free_context(&codec_ctx); avformat_close_input(&format_ctx); return AudioSegment(); } if (avcodec_open2(codec_ctx, my_codec, nullptr) < 0) { std::cerr << "Error: Could not open codec." << std::endl; avcodec_free_context(&codec_ctx); avformat_close_input(&format_ctx); return AudioSegment(); } SwrContext* swr_ctx = swr_alloc(); if (!swr_ctx) { std::cerr << "Error: Could not allocate SwrContext." << std::endl; avcodec_free_context(&codec_ctx); avformat_close_input(&format_ctx); return AudioSegment(); } codec_ctx->sample_rate = 44100; av_opt_set_chlayout(swr_ctx, "in_chlayout", &codec_ctx->ch_layout, 0); av_opt_set_int(swr_ctx, "in_sample_rate", codec_ctx->sample_rate, 0); av_opt_set_sample_fmt(swr_ctx, "in_sample_fmt", codec_ctx->sample_fmt, 0); AVChannelLayout dst_ch_layout; av_channel_layout_copy(&dst_ch_layout, &codec_ctx->ch_layout); av_channel_layout_uninit(&dst_ch_layout); av_channel_layout_default(&dst_ch_layout, 2); av_opt_set_chlayout(swr_ctx, "out_chlayout", &dst_ch_layout, 0); av_opt_set_int(swr_ctx, "out_sample_rate", codec_ctx->sample_rate, 0); av_opt_set_sample_fmt(swr_ctx, "out_sample_fmt", AV_SAMPLE_FMT_S16, 0); if (swr_init(swr_ctx) < 0) { std::cerr << "Error: Failed to initialize the resampling context" << std::endl; swr_free(&swr_ctx); avcodec_free_context(&codec_ctx); avformat_close_input(&format_ctx); return AudioSegment(); } AVPacket packet; AVFrame* frame = av_frame_alloc(); if (!frame) { std::cerr << "Error: Could not allocate frame." << std::endl; swr_free(&swr_ctx); avcodec_free_context(&codec_ctx); avformat_close_input(&format_ctx); return AudioSegment(); } std::vectoroutput; bool error_happened = false; while (av_read_frame(format_ctx, &packet) >= 0) { if (packet.stream_index == audio_stream_index) { if (avcodec_send_packet(codec_ctx, &packet) == 0) { while (true) { int ret = avcodec_receive_frame(codec_ctx, frame); if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) { break; } else if (ret < 0) { error_happened = true; std::cerr << "Error: Failed to decode audio frame (code: " << ret << ")." << std::endl; break; } if (frame->pkt_size < 4) { std::cerr << "Error: Invalid PCM packet, expected at least 4 bytes, but got " << frame->pkt_size << " bytes." << std::endl; continue; } // Rescale PTS if (frame->pts != AV_NOPTS_VALUE) { frame->pts = av_rescale_q(frame->pts, codec_ctx->time_base, format_ctx->streams[audio_stream_index]->time_base); } uint8_t* output_buffer; int output_samples = av_rescale_rnd( swr_get_delay(swr_ctx, codec_ctx->sample_rate) + frame->nb_samples, codec_ctx->sample_rate, codec_ctx->sample_rate, AV_ROUND_UP); int output_buffer_size = av_samples_get_buffer_size( nullptr, 2, output_samples, AV_SAMPLE_FMT_S16, 1); output_buffer = (uint8_t*)av_malloc(output_buffer_size); if (output_buffer) { memset(output_buffer, 0, output_buffer_size); int converted_samples = swr_convert(swr_ctx, &output_buffer, output_samples, (const uint8_t**)frame->extended_data, frame->nb_samples); if (converted_samples >= 0) { int final_size = av_samples_get_buffer_size(nullptr, 2, converted_samples, AV_SAMPLE_FMT_S16, 1); output.insert(output.end(), output_buffer, output_buffer + final_size); } else { std::cerr << "Error: Failed to convert audio samples." << std::endl; } av_free(output_buffer); } else { std::cerr << "Error: Could not allocate output buffer." << std::endl; } } } else { std::cerr << "Error: Failed to send packet to codec context." << std::endl; } } av_packet_unref(&packet); } // Handle the last frame if not handled within the loop if (!error_happened && frame->nb_samples > 0) { uint8_t* output_buffer; int output_samples = av_rescale_rnd( swr_get_delay(swr_ctx, codec_ctx->sample_rate) + frame->nb_samples, codec_ctx->sample_rate, codec_ctx->sample_rate, AV_ROUND_UP); int output_buffer_size = av_samples_get_buffer_size( nullptr, 2, output_samples, AV_SAMPLE_FMT_S16, 1); output_buffer = (uint8_t*)av_malloc(output_buffer_size); if (output_buffer) { memset(output_buffer, 0, output_buffer_size); int converted_samples = swr_convert(swr_ctx, &output_buffer, output_samples, (const uint8_t**)frame->extended_data, frame->nb_samples); if (converted_samples >= 0) { int final_size = av_samples_get_buffer_size(nullptr, 2, converted_samples, AV_SAMPLE_FMT_S16, 1); output.insert(output.end(), output_buffer, output_buffer + final_size); } else { std::cerr << "Error: Failed to convert audio samples." << std::endl; } av_free(output_buffer); } else { std::cerr << "Error: Could not allocate output buffer." << std::endl; } } int frame_width = av_get_bytes_per_sample(AV_SAMPLE_FMT_S16) * 2; std::map metadata = { {"sample_width", 2}, {"frame_rate", codec_ctx->sample_rate}, {"channels", 2}, {"frame_width", frame_width} }; av_frame_free(&frame); swr_free(&swr_ctx); avcodec_free_context(&codec_ctx); avformat_close_input(&format_ctx); return AudioSegment(static_cast (output.data()), output.size(), metadata); } void log_ffmpeg_error(int ret) { char errbuf[AV_ERROR_MAX_STRING_SIZE]; av_make_error_string(errbuf, AV_ERROR_MAX_STRING_SIZE, ret); std::cerr << "FFmpeg error: " << errbuf << std::endl; } std::ofstream AudioSegment::export_segment_to_ogg(const std::string& out_f) { av_log_set_level(AV_LOG_DEBUG); AVFormatContext* format_ctx = nullptr; AVCodecContext* codec_ctx = nullptr; AVStream* stream = nullptr; const AVCodec* codec = nullptr; AVPacket* pkt = av_packet_alloc(); AVFrame* frame = av_frame_alloc(); int ret; if (!pkt || !frame) { std::cerr << "Error: Could not allocate packet or frame." << std::endl; return std::ofstream(); } std::ofstream out_file(out_f, std::ios::binary); if (!out_file.is_open()) { std::cerr << "Error: Could not open output file." << std::endl; return std::ofstream(); } ret = avformat_alloc_output_context2(&format_ctx, nullptr, "ogg", out_f.c_str()); if (ret < 0 || !format_ctx) { std::cerr << "Error: Could not allocate output format context." << std::endl; log_ffmpeg_error(ret); return std::ofstream(); } codec = avcodec_find_encoder_by_name("libvorbis"); if (!codec) { std::cerr << "Error: Could not find Vorbis codec." << std::endl; avformat_free_context(format_ctx); return std::ofstream(); } stream = avformat_new_stream(format_ctx, codec); if (!stream) { std::cerr << "Error: Could not create new stream." << std::endl; avformat_free_context(format_ctx); return std::ofstream(); } codec_ctx = avcodec_alloc_context3(codec); if (!codec_ctx) { std::cerr << "Error: Could not allocate codec context." << std::endl; avformat_free_context(format_ctx); return std::ofstream(); } codec_ctx->sample_rate = 48000; codec_ctx->ch_layout = AV_CHANNEL_LAYOUT_STEREO; codec_ctx->bit_rate = 128000; codec_ctx->sample_fmt = codec->sample_fmts ? codec->sample_fmts[0] : AV_SAMPLE_FMT_FLTP; codec_ctx->time_base = { 1, codec_ctx->sample_rate }; stream->time_base = codec_ctx->time_base; ret = avcodec_parameters_from_context(stream->codecpar, codec_ctx); if (ret < 0) { std::cerr << "Error: Could not copy codec parameters to stream." << std::endl; log_ffmpeg_error(ret); avcodec_free_context(&codec_ctx); avformat_free_context(format_ctx); return std::ofstream(); } ret = avcodec_open2(codec_ctx, codec, nullptr); if (ret < 0) { std::cerr << "Error: Could not open codec." << std::endl; log_ffmpeg_error(ret); avcodec_free_context(&codec_ctx); avformat_free_context(format_ctx); return std::ofstream(); } if (!(format_ctx->oformat->flags & AVFMT_NOFILE)) { ret = avio_open(&format_ctx->pb, out_f.c_str(), AVIO_FLAG_WRITE); if (ret < 0) { std::cerr << "Error: Could not open file for writing." << std::endl; log_ffmpeg_error(ret); avcodec_free_context(&codec_ctx); avformat_free_context(format_ctx); return std::ofstream(); } } ret = avformat_write_header(format_ctx, nullptr); if (ret < 0) { std::cerr << "Error: Could not write file header." << std::endl; log_ffmpeg_error(ret); avio_closep(&format_ctx->pb); avcodec_free_context(&codec_ctx); avformat_free_context(format_ctx); return std::ofstream(); } int total_samples = data_.size() / (av_get_bytes_per_sample(AV_SAMPLE_FMT_S16) * codec_ctx->ch_layout.nb_channels); int samples_read = 0; codec_ctx->frame_size = 1024; frame->nb_samples = codec_ctx->frame_size; while (samples_read < total_samples) { int num_samples = std::min(codec_ctx->frame_size, total_samples - samples_read); frame->nb_samples = num_samples; ret = av_frame_get_buffer(frame, 0); if (ret < 0) { std::cerr << "Error: Could not allocate audio data buffers." << std::endl; log_ffmpeg_error(ret); avcodec_free_context(&codec_ctx); avformat_free_context(format_ctx); return std::ofstream(); } std::memcpy(frame->data[0], data_.data() + samples_read * av_get_bytes_per_sample(AV_SAMPLE_FMT_S16) * codec_ctx->ch_layout.nb_channels, num_samples * av_get_bytes_per_sample(AV_SAMPLE_FMT_S16) * codec_ctx->ch_layout.nb_channels); if (num_samples < codec_ctx->frame_size) { int padding_size = codec_ctx->frame_size - num_samples; std::memset(frame->data[0] + num_samples * av_get_bytes_per_sample(AV_SAMPLE_FMT_S16) * codec_ctx->ch_layout.nb_channels, 0, padding_size * av_get_bytes_per_sample(AV_SAMPLE_FMT_S16) * codec_ctx->ch_layout.nb_channels); frame->nb_samples = codec_ctx->frame_size; } frame->pts = av_rescale_q(samples_read, { 1, codec_ctx->sample_rate }, stream->time_base); ret = avcodec_send_frame(codec_ctx, frame); if (ret < 0) { std::cerr << "Error: Error sending frame for encoding." << std::endl; log_ffmpeg_error(ret); avcodec_free_context(&codec_ctx); avformat_free_context(format_ctx); return std::ofstream(); } while (ret >= 0) { ret = avcodec_receive_packet(codec_ctx, pkt); if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF) { break; } else if (ret < 0) { std::cerr << "Error: Error receiving packet." << std::endl; log_ffmpeg_error(ret); avcodec_free_context(&codec_ctx); avformat_free_context(format_ctx); return std::ofstream(); } pkt->pts = pkt->dts = av_rescale_q(pkt->pts, codec_ctx->time_base, stream->time_base); pkt->stream_index = stream->index; if (av_interleaved_write_frame(format_ctx, pkt) < 0) { std::cerr << "Error: Error writing packet." << std::endl; log_ffmpeg_error(ret); avcodec_free_context(&codec_ctx); avformat_free_context(format_ctx); return std::ofstream(); } av_packet_unref(pkt); } samples_read += num_samples; } avcodec_send_frame(codec_ctx, nullptr); while (avcodec_receive_packet(codec_ctx, pkt) == 0) { pkt->pts = pkt->dts = av_rescale_q(pkt->pts, codec_ctx->time_base, stream->time_base); pkt->stream_index = stream->index; if (av_interleaved_write_frame(format_ctx, pkt) < 0) { std::cerr << "Error: Error writing packet." << std::endl; log_ffmpeg_error(ret); avcodec_free_context(&codec_ctx); avformat_free_context(format_ctx); return std::ofstream(); } av_packet_unref(pkt); } av_write_trailer(format_ctx); av_frame_free(&frame); av_packet_free(&pkt); avcodec_free_context(&codec_ctx); if (format_ctx->pb) { avio_closep(&format_ctx->pb); } avformat_free_context(format_ctx); return out_file; } Errors:
[file @ 000001b3b32640c0] Setting default whitelist 'file,crypto,data' [ogg @ 000001b3b31d4a00] No extradata present Error: Could not write file header. FFmpeg error: Invalid data found when processing input [AVIOContext @ 000001b3b4aa3d80] Statistics: 0 bytes written, 0 seeks, 0 writeouts
