Recherche avancée

Sur d’autres sites (4450)

• Anomalie #3260 (Nouveau) : Problème de dump sur les tables comportants des index de longueur spéci...

  28 août 2014, par b b

  Le contexte est le suivant : http://contrib.spip.net/GIS-4?debut_comments-list=@476655#forum476655

  En résumé, depuis que la table spip_gis contient des index dont la longueur est spécifiée, le système de dump génère une erreur sur ces tables. Après pas mal de recherche, je suis remonté jusqu’à spip_mysql_show_table() (raison pour laquelle ce ticket est déclaré sur le core et non le plugin dump).

  J’ai d’abord étudié ce qui se passe dans base_copier_table() de ecrire/base/dump.php :

  http://core.spip.org/projects/spip/repository/entry/spip/ecrire/base/dump.php#L536

  À ce niveau, dans $desc_source['key'] la longueur des index n’est pas renseignée. Si on compare le contenu des fichiers de cache des descriptions des tables, on observe que tmp/cache/sql_desc*.txt ne renseigne pas la longueur des index. Alors que tmp/cachesql_desc_dump*.txt est ok, la longueur des index y est présente.

  Du coup, j’en suis remonté à sql_showtable(), et plus précisemment à spip_mysql_show_table() :

  http://core.spip.org/projects/spip/repository/entry/spip/ecrire/req/mysql.php#L754

  Depuis phpmyadmin, un SHOW CREATE TABLE renvoie bien ce qu’il faut, ex :

  CREATE TABLE `spip_gis` (
   `id_gis` bigint(21) NOT NULL AUTO_INCREMENT,
   `titre` text NOT NULL,
   `descriptif` text NOT NULL,
   `lat` double DEFAULT NULL,
   `lon` double DEFAULT NULL,
   `zoom` tinyint(4) DEFAULT NULL,
   `adresse` text NOT NULL,
   `pays` text NOT NULL,
   `code_pays` varchar(255) NOT NULL DEFAULT ’’,
   `region` text NOT NULL,
   `departement` text NOT NULL,
   `ville` text NOT NULL,
   `code_postal` varchar(255) NOT NULL DEFAULT ’’,
   PRIMARY KEY (`id_gis`),
   KEY `lat` (`lat`),
   KEY `lon` (`lon`),
   KEY `pays` (`pays`(500)),
   KEY `code_pays` (`code_pays`),
   KEY `region` (`region`(500)),
   KEY `ville` (`ville`(500)),
   KEY `code_postal` (`code_postal`),
   KEY `departement` (`departement`(500))
  ) ENGINE=MyISAM AUTO_INCREMENT=24 DEFAULT CHARSET=latin1
  

  Il semble que la regex de la ligne 736 ne match pas la table spip_gis certainement à cause des (500) :

  http://core.spip.org/projects/spip/repository/entry/spip/ecrire/req/mysql.php#L736

  Et du coup, on bascule sur le plan B, qui ne fait qu’un simple SHOW COLUMNS FROM ne contenant pas l’information de longueur des index :

  http://core.spip.org/projects/spip/repository/entry/spip/ecrire/req/mysql.php#L765

  Wala où j’en suis ^^ Je ne sais pas si ce comportement est voulu, mais il pose un sacré problème pour les dumps.

• Hikvision NVR video conversion ffmpeg

  6 août 2024, par astralmaster

  I have a Hikvision NVR that stores security camera footage that I need to display on a website. I know that Hikvision uses proprietary H.264 codec that makes it impossible to play (coherently) in popular video players, like VLC, unless you install that codec everywhere you play it.

  



  My plan was to transcode the video using ffmpeg to regular H.264 codec and AAC for audio but the produced file has the same issues as the original - no audio when playing and very disruptive video. 
So the question is, does ffmpeg support encoding from Hikvision video/audio codecs ? Or perhaps should try to convert to different web-capable codecs using ffmpeg ?
My ffmpeg command looks like this :

  



  ffmpeg -i C:\1.mp4  -c:v libx264 -preset fast -crf 30 -b:v 200k -c:a aac -strict experimental -movflags faststart -threads 0 C:\2.mp4


  



  EDIT : What's interesting is that ffplay.exe opens and plays the original video files with no problem whatsoever, even on a computer where Hikvision codecs are not isntalled, therefore I figured conversion should be possible as well ?

  



  Mediainfo output of the video file in question :

  



  General
CompleteName                     : C:\DownLoad\1.mp4
Format                           : MPEG-PS
FileSize/String                  : 8.60 MiB
Duration/String                  : 2 h 7 min
OverallBitRate/String            : 9 395 b/s
FileExtension_Invalid            : mpeg mpg m2p vob pss evo

Video
ID/String                        : 224 (0xE0)
Format                           : AVC
Format/Info                      : Advanced Video Codec
Format_Profile                   : Baseline@L4
Format_Settings                  : 1 Ref Frames
Format_Settings_CABAC/String     : No
Format_Settings_RefFrames/String : 1 frame
Format_Settings_GOP              : M=1, N=30
Duration/String                  : 2 min 0 s
Width/String                     : 1 920 pixels
Height/String                    : 1 080 pixels
DisplayAspectRatio/String        : 16:9
FrameRate_Mode/String            : Variable
ColorSpace                       : YUV
ChromaSubsampling/String         : 4:2:0
BitDepth/String                  : 8 bits
ScanType/String                  : Progressive

Audio
ID/String                        : 192 (0xC0)
Format                           : MPEG Audio
Duration/String                  : 2 h 7 min
Compression_Mode/String          : Lossy
Video_Delay/String               : -33 min 40 s


  



  Output of ffmpeg :

  



  C:\ffmpeg\bin>ffmpeg -i C:\DownLoad\1.mp4  -c:v libx264 -preset fast -crf 30 -b:v 75k -c:a aac -strict experimental -movflags faststart -threads 0 C:\DownLoad\2.mp4
ffmpeg version N-86537-gae6f6d4 Copyright (c) 2000-2017 the FFmpeg developers
  built with gcc 7.1.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-cuda --enable-cuvid --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-zlib
  libavutil      55. 66.100 / 55. 66.100
  libavcodec     57. 99.100 / 57. 99.100
  libavformat    57. 73.100 / 57. 73.100
  libavdevice    57.  7.100 / 57.  7.100
  libavfilter     6. 94.100 /  6. 94.100
  libswscale      4.  7.101 /  4.  7.101
  libswresample   2.  8.100 /  2.  8.100
  libpostproc    54.  6.100 / 54.  6.100
Input #0, mpeg, from 'C:\DownLoad\1.mp4':
  Duration: 02:07:57.93, start: 789.820800, bitrate: 9 kb/s
    Stream #0:0[0x1e0]: Video: h264 (Baseline), yuv420p(progressive), 1920x1080, 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x1c0]: Audio: pcm_mulaw, 8000 Hz, mono, s16, 64 kb/s
File 'C:\DownLoad\2.mp4' already exists. Overwrite ? [y/N] y
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (pcm_mulaw (native) -> aac (native))
Press [q] to stop, [?] for help
[aac @ 0000000002cd0280] Too many bits 8832.000000 > 6144 per frame requested, clamping to max
[libx264 @ 0000000002514c80] using cpu capabilities: MMX2 SSE2Fast LZCNT SSSE3 SSE4.2 AVX XOP FMA4
[libx264 @ 0000000002514c80] profile High, level 4.0
[libx264 @ 0000000002514c80] 264 - core 150 r2833 df79067 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=2 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=6 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=12 lookahead_threads=2 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=1 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=30 rc=crf mbtree=1 crf=30.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'C:\DownLoad\2.mp4':
  Metadata:
    encoder         : Lavf57.73.100
    Stream #0:0: Video: h264 (libx264) ([33][0][0][0] / 0x0021), yuv420p, 1920x1080, q=-1--1, 75 kb/s, 25 fps, 12800 tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.99.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/75000 buffer size: 0 vbv_delay: -1
    Stream #0:1: Audio: aac (LC) ([64][0][0][0] / 0x0040), 8000 Hz, mono, fltp, 48 kb/s
    Metadata:
      encoder         : Lavc57.99.100 aac
[mp4 @ 00000000010e9e00] Starting second pass: moving the moov atom to the beginning of the file speed= 116x
frame= 3269 fps= 66 q=-1.0 Lsize=   11086kB time=01:34:24.38 bitrate=  16.0kbits/s dup=269 drop=0 speed= 115x
video:10429kB audio:592kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.594114%
[libx264 @ 0000000002514c80] frame I:14    Avg QP:21.86  size: 59795
[libx264 @ 0000000002514c80] frame P:833   Avg QP:24.81  size:  8993
[libx264 @ 0000000002514c80] frame B:2422  Avg QP:28.70  size:   970
[libx264 @ 0000000002514c80] consecutive B-frames:  1.0%  0.2%  1.4% 97.4%
[libx264 @ 0000000002514c80] mb I  I16..4: 18.9% 66.3% 14.8%
[libx264 @ 0000000002514c80] mb P  I16..4:  4.0%  7.7%  0.4%  P16..4: 16.2%  2.0%  0.6%  0.0%  0.0%    skip:69.1%
[libx264 @ 0000000002514c80] mb B  I16..4:  0.6%  0.2%  0.0%  B16..8:  5.5%  0.1%  0.0%  direct: 0.7%  skip:92.9%  L0:44.0% L1:55.0% BI: 1.0%
[libx264 @ 0000000002514c80] 8x8 transform intra:59.0% inter:83.3%
[libx264 @ 0000000002514c80] coded y,uvDC,uvAC intra: 25.3% 36.1% 7.7% inter: 1.0% 2.3% 0.1%
[libx264 @ 0000000002514c80] i16 v,h,dc,p: 23% 24% 43% 10%
[libx264 @ 0000000002514c80] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 37% 26% 23%  2%  2%  3%  2%  3%  3%
[libx264 @ 0000000002514c80] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 43% 23% 12%  4%  4%  5%  4%  4%  2%
[libx264 @ 0000000002514c80] i8c dc,h,v,p: 81%  7%  9%  3%
[libx264 @ 0000000002514c80] Weighted P-Frames: Y:1.0% UV:0.0%
[libx264 @ 0000000002514c80] ref P L0: 73.6% 26.4%
[libx264 @ 0000000002514c80] ref B L0: 80.9% 19.1%
[libx264 @ 0000000002514c80] ref B L1: 90.0% 10.0%
[libx264 @ 0000000002514c80] kb/s:653.30
[aac @ 0000000002cd0280] Qavg: 64512.656

C:\ffmpeg\bin>


  



  Download link to sample :

  



  https://www.dropbox.com/s/9ccptsuiqk2ntsv/1.zip?dl=0

  



  This sample is exactly 2 minutes long, but VLC will tell you otherwise.

  


• Server Move For multimedia.cx

  1er août 2014, par Multimedia Mike — General

  I made a big change to multimedia.cx last week : I moved hosting from a shared web hosting plan that I had been using for 10 years to a dedicated virtual private server (VPS). In short, I now have no one to blame but myself for any server problems I experience from here on out.

  The tipping point occurred a few months ago when my game music search engine kept breaking regardless of what technology I was using. First, I had an admittedly odd C-based CGI solution which broke due to mysterious binary compatibility issues, the sort that are bound to occur when trying to make a Linux binary run on heterogeneous distributions. The second solution was an SQLite-based solution. Like the first solution, this worked great until it didn’t work anymore. Something else mysteriously broke vis-à-vis PHP and SQLite on my server. I started investigating a MySQL-based full text search solution but couldn’t make it work, and decided that I shouldn’t have to either.

  Ironically, just before I finished this entire move operation, I noticed that my SQLite-based FTS solution was working again on the old shared host. I’m not sure when that problem went away. No matter, I had already thrown the switch.

  How Hard Could It Be ?
  We all have thresholds for the type of chores we’re willing to put up with and which we’d rather pay someone else to perform. For the past 10 years, I felt that administering a website’s underlying software is something that I would rather pay someone else to worry about. To be fair, 10 years ago, I don’t think VPSs were a thing, or at least a viable thing in the consumer space, and I wouldn’t have been competent enough to properly administer one. Though I would have been a full-time Linux user for 5 years at that point, I was still the type to build all of my own packages from source (I may have still been running Linux From Scratch 10 years ago) which might not be the most tractable solution for server stability.

  These days, VPSs are a much more affordable option (easily competitive with shared web hosting). I also realized I know exactly how to install and configure all the software that runs the main components of the various multimedia.cx sites, having done it on local setups just to ensure that my automated backups would actually be useful in the event of catastrophe.

  All I needed was the will to do it.

  The Switchover Process
  Here’s the rough plan :

  • Investigate options for both VPS providers and mail hosts– I might be willing to run a web server but NOT a mail server
  • Start plotting several months in advance of my yearly shared hosting renewal date
  • Screw around for several months, playing video games and generally finding reasons to put off the move
  • Panic when realizing there are only a few days left before the yearly renewal comes due

  So that’s the planning phase. BTW, I chose Digital Ocean for VPS and Zoho for email hosting. Here’s the execution phase I did last week :

  • Register with Digital Ocean and set up DNS entries to point to the old shared host for the time being
  • Once the D-O DNS servers respond correctly using a manual ‘dig’ command, use their servers as the authoritative ones for multimedia.cx
  • Create a new Droplet (D-O VPS), install all the right software, move the databases, upload the files ; and exhaustively document each step, gotcha, and pitfall ; treat a VPS as necessarily disposable and have an eye towards iterating the process with a new VPS
  • Use /etc/hosts on a local machine to point DNS to the new server and verify that each site is working correctly
  • After everything looks all right, update the DNS records to point to the new server

  Finally, flip the switch on the MX record by pointing it to the new email provider.

  Improvements and Problems
  Hosting on Digital Ocean is quite amazing so far. Maybe it’s the SSDs. Whatever it is, all the sites are performing far better than on the old shared web host. People who edit the MultimediaWiki report that changes get saved in less than the 10 or so seconds required on the old server.

  Again, all problems are now my problems. A sore spot with the shared web host was general poor performance. The hosting company would sometimes complain that my sites were using too much CPU. I would have loved to try to optimize things. However, the cPanel interface found on many shared hosts don’t give you a great deal of data for debugging performance problems. However, same sites, same software, same load on the VPS is considerably more performant.

  Problem : I’ve already had the MySQL database die due to a spike in usage. I had to manually restart it. I was considering a cron-based solution to check if the server is running and restart it if not. In response to my analysis that my databases are mostly read and not often modified, so db crashes shouldn’t be too disastrous, a friend helpfully reminded me that, “You would not make a good sysadmin with attitudes like ‘an occasional crash is okay’.”

  To this end, I am planning to migrate the database server to a separate VPS. This is a strategy that even Digital Ocean recommends. I’m hoping that the MySQL server isn’t subject to such memory spikes, but I’ll continue to monitor it after I set it up.

  Overall, the server continues to get modest amounts of traffic. I predict it will remain that way unless Dark Shikari resurrects the x264dev blog. The biggest spike that multimedia.cx ever saw was when Steve Jobs linked to this WebM post.

  Dropped Sites
  There are a bunch of subdomains I dropped because I hadn’t done anything with them for years and I doubt anyone will notice they’re gone. One notable section that I decided to drop is the samples.mplayerhq.hu archive. It will live on, but it will be hosted by samples.ffmpeg.org, which had a full mirror anyway. The lower-end VPS instances don’t have the 53 GB necessary.

  Going Forward
  Here’s to another 10 years of multimedia.cx, even if multimedia isn’t as exciting as it was 10 years ago (personal opinion ; I’ll have another post on this later). But at least I can get working on some other projects now that this is done. For the past 4 months or so, whenever I think of doing some other project, I always remembered that this server move took priority over everything else.