Recherche avancée
Autres articles (84)
-
Les autorisations surchargées par les plugins
27 avril 2010, parMediaspip core
autoriser_auteur_modifier() afin que les visiteurs soient capables de modifier leurs informations sur la page d’auteurs -
Personnaliser les catégories
21 juin 2013, parFormulaire de création d’une catégorie
Pour ceux qui connaissent bien SPIP, une catégorie peut être assimilée à une rubrique.
Dans le cas d’un document de type catégorie, les champs proposés par défaut sont : Texte
On peut modifier ce formulaire dans la partie :
Administration > Configuration des masques de formulaire.
Dans le cas d’un document de type média, les champs non affichés par défaut sont : Descriptif rapide
Par ailleurs, c’est dans cette partie configuration qu’on peut indiquer le (...) -
Des sites réalisés avec MediaSPIP
2 mai 2011, parCette page présente quelques-uns des sites fonctionnant sous MediaSPIP.
Vous pouvez bien entendu ajouter le votre grâce au formulaire en bas de page.
Sur d’autres sites (11409)
-
Video encoding task not working with Django Celery Redis FFMPEG and GraphQL
18 juin 2023, par phanioI'm having a hard time trying to understand how is this FFMPEG encoding works while using Django, Celery, Redis, GraphQL and Docker too.


I have this video / courses platform project and want I'm trying to do using FFMPEG, Celery and Redis is to create different video resolutions so I can display them the way Youtube does inside the videoplayer ( the videoplayer is handled in frontend by Nextjs and Apollo Client ), now on the backend I've just learned that in order to use properly the FFMPEG to resize the oridinal video size, I need to use Celery and Redis to perform asyncronus tasks. I've found a few older posts here on stackoverflow and google, but is not quite enough info for someone who is using the ffmpeg and clery and redis for the first time ( I've started already step by step and created that example that adds two numbers together with celery, that works well ). Now I'm not sure what is wrong with my code, because first of all I'm not really sure where should I trigger the task from, I mean from which file, because at the end of the task I want to send the data through api using GrapQL Strawberry.


This is what I've tried by now :


So first things first my project structure looks like this


- backend #root directory
 --- backend
 -- __init__.py
 -- celery.py
 -- settings.py
 -- urls.py
 etc..

 --- static
 -- videos

 --- video
 -- models.py
 -- schema.py
 -- tasks.py
 -- types.py
 etc..

 --- .env

 --- db.sqlite3

 --- docker-compose.yml

 --- Dockerfile

 --- manage.py

 --- requirements.txt


here is my settings.py file :


from pathlib import Path
import os

# Build paths inside the project like this: BASE_DIR / 'subdir'.
BASE_DIR = Path(__file__).resolve().parent.parent

DEBUG = True

ALLOWED_HOSTS=["localhost", "0.0.0.0", "127.0.0.1"]

DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'


# Application definition

INSTALLED_APPS = [
 "corsheaders",
 'django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',

 "strawberry.django",
 "video"
]

etc...

STATIC_URL = '/static/'
MEDIA_URL = '/videos/'

STATICFILES_DIRS = [
 BASE_DIR / 'static',
 # BASE_DIR / 'frontend/build/static',
]

MEDIA_ROOT = BASE_DIR / 'static/videos'

STATIC_ROOT = BASE_DIR / 'staticfiles'

STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage'

CORS_ALLOW_ALL_ORIGINS = True


CELERY_BEAT_SCHEDULER = 'django_celery_beat.schedulers:DatabaseScheduler'

# REDIS CACHE
CACHES = {
 "default": {
 "BACKEND": "django_redis.cache.RedisCache",
 "LOCATION": f"redis://127.0.0.1:6379/1",
 "OPTIONS": {
 "CLIENT_CLASS": "django_redis.client.DefaultClient",
 },
 }
}

# Docker
CELERY_BROKER_URL = os.environ.get("CELERY_BROKER", "redis://redis:6379/0")
CELERY_RESULT_BACKEND = os.environ.get("CELERY_BROKER", "redis://redis:6379/0")


This is my main urls.py file :


from django.contrib import admin
from django.conf import settings
from django.conf.urls.static import static
from django.urls import path
from django.urls.conf import include
from strawberry.django.views import GraphQLView

from video.schema import schema

urlpatterns = [
 path('admin/', admin.site.urls),
 path("graphql", GraphQLView.as_view(schema=schema)),
]

if settings.DEBUG:
 urlpatterns += static(settings.MEDIA_URL,
 document_root=settings.MEDIA_ROOT)
 urlpatterns += static(settings.STATIC_URL,
 document_root=settings.STATIC_ROOT)


This is my celery.py file :


from __future__ import absolute_import, unicode_literals
import os
from celery import Celery
from django.conf import settings

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'backend.settings')

backend = Celery('backend')

backend.config_from_object('django.conf:settings', namespace="CELERY")

backend.autodiscover_tasks()

@backend.task(bind=True)
def debug_task(self):
 print('Request: {0!r}'.format(self.request))


This is my init.py file :


from .celery import backend as celery_backend

__all__ = ('celery_backend',)


This is my Dockerfile :


FROM python:3
ENV PYTHONUNBUFFERED=1

WORKDIR /usr/src/backend

RUN apt-get -y update
RUN apt-get -y upgrade
RUN apt-get install -y ffmpeg

COPY requirements.txt ./
RUN pip install -r requirements.txt


This is my docker-compose.yml file :


version: "3.8"

services:
 django:
 build: .
 container_name: django
 command: python manage.py runserver 0.0.0.0:8000
 volumes:
 - .:/usr/src/backend/
 ports:
 - "8000:8000"
 environment:
 - DEBUG=1
 - DJANGO_ALLOWED_HOSTS=localhost 127.0.0.1 [::1]
 - CELERY_BROKER=redis://redis:6379/0
 - CELERY_BACKEND=redis://redis:6379/0
 depends_on:
 - pgdb
 - redis

 celery:
 build: .
 command: celery -A backend worker -l INFO
 volumes:
 - .:/usr/src/backend
 depends_on:
 - django
 - redis

 pgdb:
 image: postgres
 container_name: pgdb
 environment:
 - POSTGRES_DB=postgres
 - POSTGRES_USER=postgres
 - POSTGRES_PASSWORD=postgres
 volumes:
 - pgdata:/var/lib/postgresql/data/

 redis:
 image: "redis:alpine"

volumes:
 pgdata:


And now inside my video app folder :


My models.py file :


- 

- here I've created separated fields for all resolution sizes, from video_file_2k to video_file_144, I was thinking that maybe after the process of the encoding this will populate those fields..




from django.db import models
from django.urls import reverse


class Video(models.Model):
 video_id = models.AutoField(primary_key=True, editable=False)
 slug = models.SlugField(max_length=255)
 title = models.CharField(max_length=150, blank=True, null=True)
 description = models.TextField(blank=True, null=True)
 video_file = models.FileField(null=False, blank=False)
 video_file_2k = models.FileField(null=True, blank=True)
 video_file_fullhd = models.FileField(null=True, blank=True)
 video_file_hd = models.FileField(null=True, blank=True)
 video_file_480 = models.FileField(null=True, blank=True)
 video_file_360 = models.FileField(null=True, blank=True)
 video_file_240 = models.FileField(null=True, blank=True)
 video_file_144 = models.FileField(null=True, blank=True)
 category = models.CharField(max_length=64, blank=False, null=False)
 created_at = models.DateTimeField(
 ("Created at"), auto_now_add=True, editable=False)
 updated_at = models.DateTimeField(("Updated at"), auto_now=True)

 class Meta:
 ordering = ("-created_at",)
 verbose_name = ("Video")
 verbose_name_plural = ("Videos")

 def get_absolute_url(self):
 return reverse("store:video_detail", args=[self.slug])

 def __str__(self):
 return self.title


This is my schema.py file :


import strawberry
from strawberry.file_uploads import Upload
from typing import List
from .types import VideoType
from .models import Video
from .tasks import task_video_encoding_1080p, task_video_encoding_720p


@strawberry.type
class Query:
 @strawberry.field
 def videos(self, category: str = None) -> List[VideoType]:
 if category:
 videos = Video.objects.filter(category=category)
 return videos
 return Video.objects.all()

 @strawberry.field
 def video(self, slug: str) -> VideoType:
 if slug == slug:
 video = Video.objects.get(slug=slug)
 return video

 @strawberry.field
 def video_by_id(self, video_id: int) -> VideoType:
 if video_id == video_id:
 video = Video.objects.get(pk=video_id)

 # Here I've tried to trigger my tasks, when I visited 0.0.0.0:8000/graphql url
 # and I was querying for a video by it's id , then I've got the error from celery 
 task_video_encoding_1080p.delay(video_id)
 task_video_encoding_720p.delay(video_id)

 return video


@strawberry.type
class Mutation:
 @strawberry.field
 def create_video(self, slug: str, title: str, description: str, video_file: Upload, video_file_2k: str, video_file_fullhd: str, video_file_hd: str, video_file_480: str, video_file_360: str, video_file_240: str, video_file_144: str, category: str) -> VideoType:

 video = Video(slug=slug, title=title, description=description,
 video_file=video_file, video_file_2k=video_file_2k, video_file_fullhd=video_file_fullhd, video_file_hd=video_file_hd, video_file_480=video_file_480, video_file_360=video_file_360, video_file_240=video_file_240, video_file_144=video_file_144,category=category)
 
 video.save()
 return video

 @strawberry.field
 def update_video(self, video_id: int, slug: str, title: str, description: str, video_file: str, category: str) -> VideoType:
 video = Video.objects.get(video_id=video_id)
 video.slug = slug
 video.title = title
 video.description = description
 video.video_file = video_file
 video.category = category
 video.save()
 return video

 @strawberry.field
 def delete_video(self, video_id: int) -> bool:
 video = Video.objects.get(video_id=video_id)
 video.delete
 return True


schema = strawberry.Schema(query=Query, mutation=Mutation)


This is my types.py file ( strawberry graphql related ) :


import strawberry

from .models import Video


@strawberry.django.type(Video)
class VideoType:
 video_id: int
 slug: str
 title: str
 description: str
 video_file: str
 video_file_2k: str
 video_file_fullhd: str
 video_file_hd: str
 video_file_480: str
 video_file_360: str
 video_file_240: str
 video_file_144: str
 category: str


And this is my tasks.py file :


from __future__ import absolute_import, unicode_literals
import os, subprocess
from django.conf import settings
from django.core.exceptions import ValidationError
from celery import shared_task
from celery.utils.log import get_task_logger
from .models import Video
FFMPEG_PATH = os.environ["IMAGEIO_FFMPEG_EXE"] = "/opt/homebrew/Cellar/ffmpeg/6.0/bin/ffmpeg"

logger = get_task_logger(__name__)


# CELERY TASKS
@shared_task
def add(x,y):
 return x + y


@shared_task
def task_video_encoding_720p(video_id):
 logger.info('Video Processing started')
 try:
 video = Video.objects.get(video_id=video_id)
 input_file_path = video.video_file.path
 input_file_url = video.video_file.url
 input_file_name = video.video_file.name

 # get the filename (without extension)
 filename = os.path.basename(input_file_url)

 # path to the new file, change it according to where you want to put it
 output_file_name = os.path.join('videos', 'mp4', '{}.mp4'.format(filename))
 output_file_path = os.path.join(settings.MEDIA_ROOT, output_file_name)

 # 2-pass encoding
 for i in range(1):
 new_video_720p = subprocess.call([FFMPEG_PATH, '-i', input_file_path, '-s', '1280x720', '-vcodec', 'mpeg4', '-acodec', 'libvo_aacenc', '-b', '10000k', '-pass', i, '-r', '30', output_file_path])
 # new_video_720p = subprocess.call([FFMPEG_PATH, '-i', input_file_path, '-s', '{}x{}'.format(height * 16/9, height), '-vcodec', 'mpeg4', '-acodec', 'libvo_aacenc', '-b', '10000k', '-pass', i, '-r', '30', output_file_path])

 if new_video_720p == 0:
 # save the new file in the database
 # video.video_file_hd.name = output_file_name
 video.save(update_fields=['video_file_hd'])
 logger.info('Video Processing Finished')
 return video

 else:
 logger.info('Proceesing Failed.') # Just for now

 except:
 raise ValidationError('Something went wrong')


@shared_task
# def task_video_encoding_1080p(video_id, height):
def task_video_encoding_1080p(video_id):
 logger.info('Video Processing started')
 try:
 video = Video.objects.get(video_id=video_id)
 input_file_path = video.video_file.url
 input_file_name = video.video_file.name

 # get the filename (without extension)
 filename = os.path.basename(input_file_path)

 # path to the new file, change it according to where you want to put it
 output_file_name = os.path.join('videos', 'mp4', '{}.mp4'.format(filename))
 output_file_path = os.path.join(settings.MEDIA_ROOT, output_file_name)

 for i in range(1):
 new_video_1080p = subprocess.call([FFMPEG_PATH, '-i', input_file_path, '-s', '1920x1080', '-vcodec', 'mpeg4', '-acodec', 'libvo_aacenc', '-b', '10000k', '-pass', i, '-r', '30', output_file_path])

 if new_video_1080p == 0:
 # save the new file in the database
 # video.video_file_hd.name = output_file_name
 video.save(update_fields=['video_file_fullhd'])
 logger.info('Video Processing Finished')
 return video
 else:
 logger.info('Proceesing Failed.') # Just for now

 except:
 raise ValidationError('Something went wrong')


In my first attempt I wasn't triggering the tasks no where, then I've tried to trigger the task from the schema.py file from graphql inside the video_by_id, but there I've got this error :


backend-celery-1 | django.core.exceptions.ValidationError: ['Something went wrong']
backend-celery-1 | [2023-06-18 16:38:52,859: ERROR/ForkPoolWorker-4] Task video.tasks.task_video_encoding_1080p[d33b1a42-5914-467c-ad5c-00565bc8be6f] raised unexpected: ValidationError(['Something went wrong'])
backend-celery-1 | Traceback (most recent call last):
backend-celery-1 | File "/usr/src/backend/video/tasks.py", line 81, in task_video_encoding_1080p
backend-celery-1 | new_video_1080p = subprocess.call([FFMPEG_PATH, '-i', input_file_path, '-s', '1920x1080', '-vcodec', 'mpeg4', '-acodec', 'libvo_aacenc', '-b', '10000k', '-pass', i, '-r', '30', output_file_path])
backend-celery-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-celery-1 | File "/usr/local/lib/python3.11/subprocess.py", line 389, in call
backend-celery-1 | with Popen(*popenargs, **kwargs) as p:
backend-celery-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-celery-1 | File "/usr/local/lib/python3.11/subprocess.py", line 1026, in __init__
backend-celery-1 | self._execute_child(args, executable, preexec_fn, close_fds,
backend-celery-1 | File "/usr/local/lib/python3.11/subprocess.py", line 1883, in _execute_child
backend-celery-1 | self.pid = _fork_exec(
backend-celery-1 | ^^^^^^^^^^^
backend-celery-1 | TypeError: expected str, bytes or os.PathLike object, not int
backend-celery-1 | 
backend-celery-1 | During handling of the above exception, another exception occurred:
backend-celery-1 | 
backend-celery-1 | Traceback (most recent call last):
backend-celery-1 | File "/usr/local/lib/python3.11/site-packages/celery/app/trace.py", line 477, in trace_task
backend-celery-1 | R = retval = fun(*args, **kwargs)
backend-celery-1 | ^^^^^^^^^^^^^^^^^^^^
backend-celery-1 | File "/usr/local/lib/python3.11/site-packages/celery/app/trace.py", line 760, in __protected_call__
backend-celery-1 | return self.run(*args, **kwargs)
backend-celery-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^
backend-celery-1 | File "/usr/src/backend/video/tasks.py", line 93, in task_video_encoding_1080p
backend-celery-1 | raise ValidationError('Something went wrong')
backend-celery-1 | django.core.exceptions.ValidationError: ['Something went wrong']


If anyone has done this kind of project or something like this please any suggestion or help is much appreciated.


Thank you in advance !


-
To all Matomo plugin developers : Matomo 5 is coming, make your plugin compatible now
5 mai 2023, par Matomo Core Team — DevelopmentWe’re planning to release the first beta of Matomo 5 in a few weeks. For making it easy for Matomo users to be able to upgrade to this beta, it would be great to have as many plugins on the Marketplace as possible already updated and compatible with Matomo 5. Then many users would be able to upgrade to the first beta without any issues.
Presumably, as you put your plugin on our Marketplace, you want people to use it. Making your plugin compatible with Matomo 5 helps ensure that people will be able to find and keep using your plugin. If your plugin is not compatible with Matomo 5, your plugin will be automatically deactivated in Matomo 5 instances. We’ll be happy to help you achieve compatibility should there be any issue.
How do I upgrade my Matomo instance to Matomo 5 ?
If you have installed your Matomo development environment through git, you can simply checkout the Matomo 5 branch “
5.x-dev” and install its dependencies by executing these commands :git checkout 5.x-devcomposer install
Alternatively, you can also download the latest version directly from GitHub as a zip file and run
composer installafterwards.How do I upgrade my plugin to Matomo 5 ?
- We recommend you create a new branch for your plugin that supports Matomo 5. For example a branch named “
5.x-dev“. This way you will be able to make changes to your plugin for Matomo 4 and Matomo 5 and release separate versions. - Check out our migration guide from Matomo 4 to Matomo 5 and make sure to review the list of all breaking changes.
- Release a new version of your plugin in GitHub as usual.
While there are some breaking changes in Matomo 5, most of our Platform APIs remain unchanged, and almost all changes are for rarely used APIs. Quite often, making your plugin compatible with Matomo 5 will just be a matter of adjusting the “
plugin.json” file (as mentioned in the migration guide).You can find all developer documentation on our developer zone which has already been updated for Matomo 5.
How do I know my plugin changes were released successfully ?
If you have configured an email address within your “
plugin.json” file, then you will receive a confirmation or an error email within a few minutes. Alternatively, you can also check out your plugin page on the Marketplace directly. If the plugin release was successful, you will see additional links below the download button showing which versions your plugin is compatible with.
How can switch between Matomo 4 and Matomo 5 or downgrade to Matomo 4 ?
To downgrade from Matomo 5 to Matomo 4 in your Matomo development environment :
- check out the “
4.x-dev” branch - run “
composer install” as usual
When will the final Matomo 5 release be available ?
We estimate the final stable Matomo 5.0.0 release will be released in approx. 2-3 months.
What is new in Matomo 5 ?
We don’t have a summary of the changes available just yet but you can see all closed issues within this release here.
Any questions or need help ?
If you have any questions, or experience any problems during the migration, don’t hesitate to get in touch with us. We’ll be happy to help get your plugin compatible and the update published. If you find any undocumented breaking change or find any step during the migration process not clear, please let us know as well.
Thank you for contributing a plugin to the Marketplace and making Matomo better. We really appreciate your work !
-
FFmpeg invalid data found when processing input with D3D11VA and DXVA2 hw acceleration
6 mai 2023, par grill2010I'm currently porting my Android streaming app to Windows and for decoding the h264 video stream I use FFmpeg with possible hardware acceleration. Last two weeks I was reading a lot of documentation and studied a lot of examples on the internet. For my project I use JavaCV which is internally using FFmpeg 5.1.2. On Windows I support D3D11VA, DXVA2 and Cuvid for hardware acceleration (and software decoding as fallback). During testing I noticed that I get some strange artefacts when using D3D11VA or DXVA2 hw acceleration. Upon further investigation I saw that I receive a lot of


"Invalid data found when processing input"


errors when calling
avcodec_send_packet. It seems this error only occurs on certain key frames. The error is reproducable all the time. The software decoder or cuvid decoder has absolutely no problem to process and to decode such a frame, so not sure why there should be an invalid data in the frame ? I played around a lot with the decoder configuration but nothing seems to help and at that point I think this is definitely not normal behaviour.

I provided a reproducable example which can be downloaded from here. All the important part is in the App.java class. In addition an example of the code was posted below. The example is trying to decode a key frame. The keyframe data with sps and pps is read from a file in the resource folder of the project.


To run the project just perform a .\gradlew build and afterwards a .\gradlew run. If you run the example the last log message shown in the terminal should be "SUCESS with HW decoding". The hardware decoder can be changed via the HW_DEVICE_TYPE variable in the App.java class. To disable hw acceleration just set the USE_HW_ACCEL to false.


For me everything seems to be correct and I have no idea what could be wrong with the code. I looked a lot on the internet to find the root cause of the issue and I did not really found a solution but other sources which are related to (maybe) the same problem


https://www.mail-archive.com/libav-user@ffmpeg.org/...


https://stackoverflow.com/questions/67307397/ffmpeg-...


I also found another streaming app on Windows which can use D3D11VA and DXVA2 hardware acceleration called Chiaki (it requires a PS4 or a PS5) which seems to have the exact same problem. I used the build provided here. It will fail to decode certain key frames as well when hardware acceleration with D3D11VA or DXVA2 is selected (e.g. the first key frame received by the stream). Chiaki can output the seemingly faulty frame but this is also possible with my example by setting the USE_AV_EF_EXPLODE to false.


Are there any ffmpeg gurus around that can check what's the problem with D3D11VA or DXVA2 ? Anything else that needs to be done to make the D3D11VA and DXVA2 hardware decoder work ? I'm now completly out of ideas and I'm not even sure if this is fixable.


I have Windows 11 installed on my test machine, and I have the latest Nvidea drivers installed.


Edit : here is a shrinked complete example of my project (keyframe file which includes sps and pps can be downloaded from here. It's a hex string file and can be decoded with the provided HexUtil class)


import javafx.application.Application;
import javafx.scene.Scene;
import javafx.scene.layout.Pane;
import javafx.stage.Stage;
import org.bytedeco.ffmpeg.avcodec.AVCodec;
import org.bytedeco.ffmpeg.avcodec.AVCodecContext;
import org.bytedeco.ffmpeg.avcodec.AVCodecHWConfig;
import org.bytedeco.ffmpeg.avcodec.AVPacket;
import org.bytedeco.ffmpeg.avutil.AVBufferRef;
import org.bytedeco.ffmpeg.avutil.AVDictionary;
import org.bytedeco.ffmpeg.avutil.AVFrame;
import org.bytedeco.javacpp.BytePointer;
import org.bytedeco.javacpp.IntPointer;
import org.bytedeco.javacv.FFmpegLogCallback;
import org.tinylog.Logger;

import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
import java.util.Objects;
import java.util.function.Consumer;

import static org.bytedeco.ffmpeg.avcodec.AVCodecContext.AV_EF_EXPLODE;
import static org.bytedeco.ffmpeg.avcodec.AVCodecContext.FF_THREAD_SLICE;
import static org.bytedeco.ffmpeg.global.avcodec.*;
import static org.bytedeco.ffmpeg.global.avutil.*;

public class App extends Application {

 /**** decoder variables ****/

 private AVHWContextInfo hardwareContext;

 private AVCodec decoder;
 private AVCodecContext m_VideoDecoderCtx;

 private AVCodecContext.Get_format_AVCodecContext_IntPointer formatCallback;
 private final int streamResolutionX = 1920;
 private final int streamResolutionY = 1080;

 // AV_HWDEVICE_TYPE_CUDA // example works with cuda
 // AV_HWDEVICE_TYPE_DXVA2 // producing Invalid data found on keyframe
 // AV_HWDEVICE_TYPE_D3D11VA // producing Invalid data found on keyframe
 private static final int HW_DEVICE_TYPE = AV_HWDEVICE_TYPE_DXVA2;

 private static final boolean USE_HW_ACCEL = true;

 private static final boolean USE_AV_EF_EXPLODE = true;

 public static void main(final String[] args) {
 //System.setProperty("prism.order", "d3d,sw");
 System.setProperty("prism.vsync", "false");
 Application.launch(App.class);
 }

 @Override
 public void start(final Stage primaryStage) {
 final Pane dummyPane = new Pane();
 dummyPane.setStyle("-fx-background-color: black");
 final Scene scene = new Scene(dummyPane, this.streamResolutionX, this.streamResolutionY);
 primaryStage.setScene(scene);
 primaryStage.show();
 primaryStage.setMinWidth(480);
 primaryStage.setMinHeight(360);

 this.initializeFFmpeg(result -> {
 if (!result) {
 Logger.error("FFmpeg could not be initialized correctly, terminating program");
 System.exit(1);
 return;
 }
 this.performTestFramesFeeding();
 });
 }

 private void initializeFFmpeg(final Consumer<boolean> finishHandler) {
 FFmpegLogCallback.setLevel(AV_LOG_DEBUG); // Increase log level until the first frame is decoded
 //FFmpegLogCallback.setLevel(AV_LOG_QUIET);
 this.decoder = avcodec_find_decoder(AV_CODEC_ID_H264); // usually decoder name is h264 and without hardware support it's yuv420p otherwise nv12

 if (this.decoder == null) {
 Logger.error("Unable to find decoder for format {}", "h264");
 finishHandler.accept(false);
 return;
 }
 Logger.info("Current decoder name: {}, {}", this.decoder.name().getString(), this.decoder.long_name().getString());

 if (true) {
 for (; ; ) {
 this.m_VideoDecoderCtx = avcodec_alloc_context3(this.decoder);
 if (this.m_VideoDecoderCtx == null) {
 Logger.error("Unable to find decoder for format AV_CODEC_ID_H264");
 if (this.hardwareContext != null) {
 this.hardwareContext.free();
 this.hardwareContext = null;
 }
 continue;
 }

 if (App.USE_HW_ACCEL) {
 this.hardwareContext = this.createHardwareContext();
 if (this.hardwareContext != null) {
 Logger.info("Set hwaccel support");
 this.m_VideoDecoderCtx.hw_device_ctx(this.hardwareContext.hwContext()); // comment to disable hwaccel
 }
 } else {
 Logger.info("Hwaccel manually disabled");
 }


 // Always request low delay decoding
 this.m_VideoDecoderCtx.flags(this.m_VideoDecoderCtx.flags() | AV_CODEC_FLAG_LOW_DELAY);

 // Allow display of corrupt frames and frames missing references
 this.m_VideoDecoderCtx.flags(this.m_VideoDecoderCtx.flags() | AV_CODEC_FLAG_OUTPUT_CORRUPT);
 this.m_VideoDecoderCtx.flags2(this.m_VideoDecoderCtx.flags2() | AV_CODEC_FLAG2_SHOW_ALL);

 if (App.USE_AV_EF_EXPLODE) {
 // Report decoding errors to allow us to request a key frame
 this.m_VideoDecoderCtx.err_recognition(this.m_VideoDecoderCtx.err_recognition() | AV_EF_EXPLODE);
 }

 // Enable slice multi-threading for software decoding
 if (this.m_VideoDecoderCtx.hw_device_ctx() == null) { // if not hw accelerated
 this.m_VideoDecoderCtx.thread_type(this.m_VideoDecoderCtx.thread_type() | FF_THREAD_SLICE);
 this.m_VideoDecoderCtx.thread_count(2/*AppUtil.getCpuCount()*/);
 } else {
 // No threading for HW decode
 this.m_VideoDecoderCtx.thread_count(1);
 }

 this.m_VideoDecoderCtx.width(this.streamResolutionX);
 this.m_VideoDecoderCtx.height(this.streamResolutionY);
 this.m_VideoDecoderCtx.pix_fmt(this.getDefaultPixelFormat());

 this.formatCallback = new AVCodecContext.Get_format_AVCodecContext_IntPointer() {
 @Override
 public int call(final AVCodecContext context, final IntPointer pixelFormats) {
 final boolean hwDecodingSupported = context.hw_device_ctx() != null && App.this.hardwareContext != null;
 final int preferredPixelFormat = hwDecodingSupported ?
 App.this.hardwareContext.hwConfig().pix_fmt() :
 context.pix_fmt();
 int i = 0;
 while (true) {
 final int currentSupportedFormat = pixelFormats.get(i++);
 System.out.println("Supported pixel formats " + currentSupportedFormat);
 if (currentSupportedFormat == preferredPixelFormat) {
 Logger.info("[FFmpeg]: pixel format in format callback is {}", currentSupportedFormat);
 return currentSupportedFormat;
 }
 if (currentSupportedFormat == AV_PIX_FMT_NONE) {
 break;
 }
 }

 i = 0;
 while (true) { // try again and search for yuv
 final int currentSupportedFormat = pixelFormats.get(i++);
 if (currentSupportedFormat == AV_PIX_FMT_YUV420P) {
 Logger.info("[FFmpeg]: Not found in first match so use {}", AV_PIX_FMT_YUV420P);
 return currentSupportedFormat;
 }
 if (currentSupportedFormat == AV_PIX_FMT_NONE) {
 break;
 }
 }

 i = 0;
 while (true) { // try again and search for nv12
 final int currentSupportedFormat = pixelFormats.get(i++);
 if (currentSupportedFormat == AV_PIX_FMT_NV12) {
 Logger.info("[FFmpeg]: Not found in second match so use {}", AV_PIX_FMT_NV12);
 return currentSupportedFormat;
 }
 if (currentSupportedFormat == AV_PIX_FMT_NONE) {
 break;
 }
 }

 Logger.info("[FFmpeg]: pixel format in format callback is using fallback {}", AV_PIX_FMT_NONE);
 return AV_PIX_FMT_NONE;
 }
 };
 this.m_VideoDecoderCtx.get_format(this.formatCallback);

 final AVDictionary options = new AVDictionary(null);
 final int result = avcodec_open2(this.m_VideoDecoderCtx, this.decoder, options);
 if (result < 0) {
 Logger.error("avcodec_open2 was not successful");
 finishHandler.accept(false);
 return;
 }
 av_dict_free(options);
 break;
 }
 }

 if (this.decoder == null || this.m_VideoDecoderCtx == null) {
 finishHandler.accept(false);
 return;
 }
 finishHandler.accept(true);
 }

 private AVHWContextInfo createHardwareContext() {
 AVHWContextInfo result = null;
 for (int i = 0; ; i++) {
 final AVCodecHWConfig config = avcodec_get_hw_config(this.decoder, i);
 if (config == null) {
 break;
 }

 if ((config.methods() & AV_CODEC_HW_CONFIG_METHOD_HW_DEVICE_CTX) < 0) {
 continue;
 }
 final int device_type = config.device_type();
 if (device_type != App.HW_DEVICE_TYPE) {
 continue;
 }
 final AVBufferRef hw_context = av_hwdevice_ctx_alloc(device_type);
 if (hw_context == null || av_hwdevice_ctx_create(hw_context, device_type, (String) null, null, 0) < 0) {
 Logger.error("HW accel not supported for type {}", device_type);
 av_free(config);
 av_free(hw_context);
 } else {
 Logger.info("HW accel created for type {}", device_type);
 result = new AVHWContextInfo(config, hw_context);
 }
 break;
 }

 return result;
 }

 @Override
 public void stop() {
 this.releaseNativeResources();
 }

 /************************/
 /*** video processing ***/
 /************************/


 private void performTestFramesFeeding() {
 final AVPacket pkt = av_packet_alloc();
 if (pkt == null) {
 return;
 }
 try (final BytePointer bp = new BytePointer(65_535 * 4)) {
 final byte[] frameData = AVTestFrames.h264KeyTestFrame;


 bp.position(0);

 bp.put(frameData);
 bp.limit(frameData.length);

 pkt.data(bp);
 pkt.capacity(bp.capacity());
 pkt.size(frameData.length);
 pkt.position(0);
 pkt.limit(frameData.length);
 final AVFrame avFrame = av_frame_alloc();

 final int err = avcodec_send_packet(this.m_VideoDecoderCtx, pkt); // this will fail with D3D11VA and DXVA2
 if (err < 0) {
 final BytePointer buffer = new BytePointer(512);
 av_strerror(err, buffer, buffer.capacity());
 final String string = buffer.getString();
 System.out.println("Error on decoding test frame " + err + " message " + string);
 av_frame_free(avFrame);
 return;
 }

 final int result = avcodec_receive_frame(this.m_VideoDecoderCtx, avFrame);
 final AVFrame decodedFrame;
 if (result == 0) {
 if (this.m_VideoDecoderCtx.hw_device_ctx() == null) {
 decodedFrame = avFrame;
 av_frame_unref(decodedFrame);
 System.out.println("SUCESS with SW decoding");
 } else {
 final AVFrame hwAvFrame = av_frame_alloc();
 if (av_hwframe_transfer_data(hwAvFrame, avFrame, 0) < 0) {
 System.out.println("Failed to transfer frame from hardware");
 av_frame_unref(hwAvFrame);
 decodedFrame = avFrame;
 } else {
 av_frame_unref(avFrame);
 decodedFrame = hwAvFrame;
 System.out.println("SUCESS with HW decoding");
 }
 av_frame_unref(decodedFrame);
 }
 } else {
 final BytePointer buffer = new BytePointer(512);
 av_strerror(result, buffer, buffer.capacity());
 final String string = buffer.getString();
 System.out.println("error " + result + " message " + string);
 av_frame_free(avFrame);
 }
 } finally {
 if (pkt.stream_index() != -1) {
 av_packet_unref(pkt);
 }
 pkt.releaseReference();
 }
 }

 final Object releaseLock = new Object();
 private volatile boolean released = false;

 private void releaseNativeResources() {
 if (this.released) {
 return;
 }
 this.released = true;
 synchronized (this.releaseLock) {
 // Close the video codec
 if (this.m_VideoDecoderCtx != null) {
 avcodec_free_context(this.m_VideoDecoderCtx);
 this.m_VideoDecoderCtx = null;
 }

 // close the format callback
 if (this.formatCallback != null) {
 this.formatCallback.close();
 this.formatCallback = null;
 }

 // close hw context
 if (this.hardwareContext != null) {
 this.hardwareContext.free();
 }
 }
 }

 private int getDefaultPixelFormat() {
 return AV_PIX_FMT_YUV420P; // Always return yuv420p here
 }

 public static final class HexUtil {
 private static final char[] hexArray = "0123456789ABCDEF".toCharArray();

 private HexUtil() {
 }

 public static String hexlify(final byte[] bytes) {
 final char[] hexChars = new char[bytes.length * 2];

 for (int j = 0; j < bytes.length; ++j) {
 final int v = bytes[j] & 255;
 hexChars[j * 2] = HexUtil.hexArray[v >>> 4];
 hexChars[j * 2 + 1] = HexUtil.hexArray[v & 15];
 }

 return new String(hexChars);
 }

 public static byte[] unhexlify(final String argbuf) {
 final int arglen = argbuf.length();
 if (arglen % 2 != 0) {
 throw new RuntimeException("Odd-length string");
 } else {
 final byte[] retbuf = new byte[arglen / 2];

 for (int i = 0; i < arglen; i += 2) {
 final int top = Character.digit(argbuf.charAt(i), 16);
 final int bot = Character.digit(argbuf.charAt(i + 1), 16);
 if (top == -1 || bot == -1) {
 throw new RuntimeException("Non-hexadecimal digit found");
 }

 retbuf[i / 2] = (byte) ((top << 4) + bot);
 }

 return retbuf;
 }
 }
 }

 public static final class AVHWContextInfo {
 private final AVCodecHWConfig hwConfig;
 private final AVBufferRef hwContext;

 private volatile boolean freed = false;

 public AVHWContextInfo(final AVCodecHWConfig hwConfig, final AVBufferRef hwContext) {
 this.hwConfig = hwConfig;
 this.hwContext = hwContext;
 }

 public AVCodecHWConfig hwConfig() {
 return this.hwConfig;
 }

 public AVBufferRef hwContext() {
 return this.hwContext;
 }

 public void free() {
 if (this.freed) {
 return;
 }
 this.freed = true;
 av_free(this.hwConfig);
 av_free(this.hwContext);
 }


 @Override
 public boolean equals(Object o) {
 if (this == o) return true;
 if (o == null || getClass() != o.getClass()) return false;
 AVHWContextInfo that = (AVHWContextInfo) o;
 return freed == that.freed && Objects.equals(hwConfig, that.hwConfig) && Objects.equals(hwContext, that.hwContext);
 }

 @Override
 public int hashCode() {
 return Objects.hash(hwConfig, hwContext, freed);
 }

 @Override
 public String toString() {
 return "AVHWContextInfo[" +
 "hwConfig=" + this.hwConfig + ", " +
 "hwContext=" + this.hwContext + ']';
 }

 }

 public static final class AVTestFrames {

 private AVTestFrames() {

 }

 static {
 InputStream inputStream = null;
 try {
 inputStream = AVTestFrames.class.getClassLoader().getResourceAsStream("h264_test_key_frame.txt");
 final byte[] h264TestFrameBuffer = inputStream == null ? new byte[0] : inputStream.readAllBytes();
 final String h264TestFrame = new String(h264TestFrameBuffer, StandardCharsets.UTF_8);
 AVTestFrames.h264KeyTestFrame = HexUtil.unhexlify(h264TestFrame);
 } catch (final IOException e) {
 Logger.error(e, "Could not parse test frame");
 } finally {
 if (inputStream != null) {
 try {
 inputStream.close();
 inputStream = null;
 } catch (final IOException e) {
 Logger.error(e, "Could not close test frame input stream");
 }
 }
 }
 }

 public static byte[] h264KeyTestFrame;
 }
}
</boolean>

The build gradle of the project looks like this


plugins {
 id 'application'
 id 'org.openjfx.javafxplugin' version '0.0.13'
}

group 'com.test.example'
version '1.0.0'

repositories {
 mavenCentral()
 mavenLocal()
 maven { url 'https://jitpack.io' }
}

dependencies {
 implementation group: 'org.bytedeco', name: 'javacv-platform', version: '1.5.8'
 implementation group: 'com.github.oshi', name: 'oshi-core', version: '3.4.3'
 implementation 'org.tinylog:tinylog-api:2.1.0'
 implementation 'org.tinylog:tinylog-impl:2.1.0'
 implementation 'org.jcodec:jcodec:0.2.5'
}

test {
 useJUnitPlatform()
}

javafx {
 version = '17.0.6'
 modules = ['javafx.graphics', 'javafx.controls', 'javafx.fxml', 'javafx.base']
}

mainClassName = 'com.test.example.App'


Syndiquer tout le site
