Recherche avancée
Médias (1)
-
Bug de détection d’ogg
22 mars 2013, par
Mis à jour : Avril 2013
Langue : français
Type : Video
Autres articles (62)
-
Websites made with MediaSPIP
2 mai 2011, parThis page lists some websites based on MediaSPIP.
-
Creating farms of unique websites
13 avril 2011, parMediaSPIP platforms can be installed as a farm, with a single "core" hosted on a dedicated server and used by multiple websites.
This allows (among other things) : implementation costs to be shared between several different projects / individuals rapid deployment of multiple unique sites creation of groups of like-minded sites, making it possible to browse media in a more controlled and selective environment than the major "open" (...) -
Gestion des droits de création et d’édition des objets
8 février 2011, parPar défaut, beaucoup de fonctionnalités sont limitées aux administrateurs mais restent configurables indépendamment pour modifier leur statut minimal d’utilisation notamment : la rédaction de contenus sur le site modifiables dans la gestion des templates de formulaires ; l’ajout de notes aux articles ; l’ajout de légendes et d’annotations sur les images ;
Sur d’autres sites (8452)
-
My python script using ffmpeg captures video content, but the captured content freezes in the middle and jumps frames
11 novembre 2022, par Supriyo MitraI am new to ffmpeg and I am trying to use it through a python script. The python functions that captures the video content is given below. The problem I am facing is that the captured content freezes at (uneven) intervals and skips a few frames every time it happens.


` def capturelivestream(self, argslist):
 streamurl, outnum, feedid, outfilename = argslist[0], argslist[1], argslist[2], argslist[3]
 try:
 info = ffmpeg.probe(streamurl, select_streams='a')
 streams = info.get('streams', [])
 except:
 streams = []
 if len(streams) == 0:
 print('There are no streams available')
 stream = {}
 else:
 stream = streams[0]
 for stream in streams:
 if stream.get('codec_type') != 'audio':
 continue
 else:
 break
 if 'channels' in stream.keys():
 channels = stream['channels']
 samplerate = float(stream['sample_rate'])
 else:
 channels = None
 samplerate = 44100
 process = ffmpeg.input(streamurl).output('pipe:', pix_fmt='yuv420p', format='avi', vcodec='libx264', acodec='pcm_s16le', ac=channels, ar=samplerate, vsync=0, loglevel='quiet').run_async(pipe_stdout=True)
 fpath = os.path.dirname(outfilename)
 fnamefext = os.path.basename(outfilename)
 fname = fnamefext.split(".")[0]
 read_size = 320 * 180 * 3 # This is width * height * 3
 lastcaptured = time.time()
 maxtries = 12
 ntries = 0
 while True:
 if process:
 inbytes = process.stdout.read(read_size)
 if inbytes is not None and inbytes.__len__() > 0:
 try:
 frame = (np.frombuffer(inbytes, np.uint8).reshape([180, 320, 3]))
 except:
 print("Failed to reshape frame: %s"%sys.exc_info()[1].__str__())
 continue # This could be an issue if there is a continuous supply of frames that cannot be reshaped
 self.processq.put([outnum, frame])
 lastcaptured = time.time()
 ntries = 0
 else:
 if self.DEBUG:
 print("Could not read frame for feed ID %s"%feedid)
 t = time.time()
 if t - lastcaptured > 30: # If the frames can't be read for more than 30 seconds...
 print("Reopening feed identified by feed ID %s"%feedid)
 process = ffmpeg.input(streamurl).output('pipe:', pix_fmt='yuv420p', format='avi', vcodec='libx264', acodec='pcm_s16le', ac=channels, ar=samplerate, vsync=0, loglevel='quiet').run_async(pipe_stdout=True)
 ntries += 1
 if ntries > maxtries:
 if self.DEBUG:
 print("Stream %s is no longer available."%streamurl)
 # DB statements removed here
 
 break # Break out of infinite loop.
 continue
 
 return None`



The function that captures the frames is as follows :



` def framewriter(self, outlist):
 isempty = False
 endofrun = False
 while True:
 frame = None
 try:
 args = self.processq.get()
 except: # Sometimes, the program crashes at this point due to lack of memory...
 print("Error in framewriter while reading from queue: %s"%sys.exc_info()[1].__str__())
 continue
 outnum = args[0]
 frame = args[1]
 if outlist.__len__() > outnum:
 out = outlist[outnum]
 else:
 if self.DEBUG == 2:
 print("Could not get writer %s"%outnum)
 continue
 if frame is not None and out is not None:
 out.write(frame)
 isempty = False
 endofrun = False
 else:
 if self.processq.empty() and not isempty:
 isempty = True
 elif self.processq.empty() and isempty: # processq queue is empty now and was empty last time
 print("processq is empty")
 endofrun = True
 elif endofrun and isempty:
 print("Could not find any frames to process. Quitting")
 break
 print("Done writing feeds. Quitting.")
 return None`


The scenario is as follows : There are multiple video streams from a certain website at any time during the day, and the program containing these functions has to capture them as they get streamed. The memory available to this program is 6GB and there could be upto 3 streams running at any instant. Given below is the relevant main section of the script that uses the functions given above.





`itftennis = VideoBot(siteurl)
outlist = []
t = Thread(target=itftennis.framewriter, args=(outlist,))
t.daemon = True
t.start()
tp = Thread(target=handleprocesstermination, args=())
tp.daemon = True
tp.start()
# Create a database connection and as associated cursor object. We will handle database operations from main thread only.
# DB statements removed from here...
feedidlist = []
vidsdict = {}
streampattern = re.compile("\?vid=(\d+)$")
while True:
 streampageurls = itftennis.checkforlivestream()
 if itftennis.DEBUG:
 print("Checking for new urls...")
 print(streampageurls.__len__())
 if streampageurls.__len__() > 0:
 argslist = []
 newurlscount = 0
 for streampageurl in streampageurls:
 newstream = False
 sps = re.search(streampattern, streampageurl)
 if sps:
 streamnum = sps.groups()[0]
 if streamnum not in vidsdict.keys(): # Check if this stream has already been processed.
 vidsdict[streamnum] = 1
 newstream = True
 else:
 continue
 else:
 continue
 print("Detected new live stream... Getting it.")
 streamurl = itftennis.getstreamurlfrompage(streampageurl)
 print("Adding %s to list..."%streamurl)
 if streamurl is not None:
 # Now, get feed metadata...
 metadata = itftennis.getfeedmetadata(streampageurl)
 if metadata is None:
 continue
 # lines to get matchescounter omitted here...
 if matchescounter >= itftennis.__class__.MAX_CONCURRENT_MATCHES:
 break
 if newstream is True:
 newurlscount += 1
 outfilename = time.strftime("./videodump/" + "%Y%m%d%H%M%S",time.localtime())+".avi"
 out = open(outfilename, "wb")
 outlist.append(out) # Save it in the list and take down the number for usage in framewriter
 outnum = outlist.__len__() - 1
 # Save metadata in DB
 # lines omitted here....
 argslist.append([streamurl, outnum, feedid, outfilename]) 
 else:
 print("Couldn't get the stream url from page")
 if newurlscount > 0:
 for args in argslist:
 try:
 p = Process(target=itftennis.capturelivestream, args=(args,))
 p.start()
 processeslist.append(p)
 if itftennis.DEBUG:
 print("Started process with args %s"%args)
 except:
 print("Could not start process due to error: %s"%sys.exc_info()[1].__str__())
 print("Created processes, continuing now...")
 continue
 time.sleep(itftennis.livestreamcheckinterval)
t.join()
tp.join()
for out in outlist:
 out.close()`





Please accept my apologies for swamping with this amount of code. I wanted to provide maximum context to my problem. I have removed the absolutely irrelevant DB statements, but apart from that this is what the code looks like.


If you need to know anything else about the code, please let me know. What I would really like to know is if I am using the ffmpeg streams capturing statements correctly. The stream contains both video and audio components and I need to capture both. Hence I am making the following call :


process = ffmpeg.input(streamurl).output('pipe:', pix_fmt='yuv420p', format='avi', vcodec='libx264', acodec='pcm_s16le', ac=channels, ar=samplerate, vsync=0, loglevel='quiet').run_async(pipe_stdout=True)


Is this how it is supposed to be done ? More importantly, why do I keep getting the freezes in the output video. I have monitored the streams manually, and they are quite consistent. Frame losses do not happen when I view them on the website (at least it is not obviously noticeable). Also, I have run 'top' command on the host running the program. The CPU usage sometimes go over 100% (which, I came to understand from some answers on SO, is to be expected when running ffmpeg) but the memory usage usually remain below 30%. So what is the issue here. What do I need to do in order to fix this problem (other than learn more about how ffmpeg works).


Thanks


I have tried using various ffmpeg options (while trying to find similar issues that others encountered). I also tried running ffmpeg from command line for a limited period of time (11 mins), using the same options as used in the python code, and the captured content came out quite well. No freezes. No jumps in frames. But I need to use it in an automated way and there would be multiple streams at any time. Also, when I try playing the captured content using ffplay, I sometimes get the message "co located POCs unavailable" when these freezes happen. What does it mean ?


-
Android player, FFMPEG, JNI
14 septembre 2022, par Nikita PavlenkoCan I use
std::asyncwithjnior I should throughstd::threadorpthread_create? Because i'm writing android player with my C++ lib that uses std::async for demuxing, video and audio. I saw that every callCallVoidMethod(of course with attach and detach thread) create new thread (thread id in Java methods is increased). And in some time a big error occurres

A/e.androidplaye: java_vm_ext.cc:594] JNI DETECTED ERROR IN APPLICATION: thread Thread[2,tid=3420,Native,Thread*=0xb400007bd4470c00,peer=0x12cc1510,"Thread-8687"] using JNIEnv* from thread Thread[3,tid=3419,Native,Thread*=0xb400007aadc2e400,peer=0x12d00000,"Thread-8688"]
 java_vm_ext.cc:594] in call to CallVoidMethodV
A/e.androidplaye: runtime.cc:675] Runtime aborting...
 runtime.cc:675] Dumping all threads without mutator lock held
 runtime.cc:675] All threads:
 runtime.cc:675] DALVIK THREADS (23):
 runtime.cc:675] "Thread-8687" prio=10 tid=2 Runnable
 runtime.cc:675] | group="" sCount=0 ucsCount=0 flags=0 obj=0x12d80000 self=0xb400007bd4470c00
 runtime.cc:675] | sysTid=3420 nice=-10 cgrp=default sched=0/0 handle=0x7aa4166cb0
 runtime.cc:675] | state=R schedstat=( 6040359820 215025421 9416 ) utm=498 stm=105 core=1 HZ=100
 runtime.cc:675] | stack=0x7aa406f000-0x7aa4071000 stackSize=991KB
 runtime.cc:675] | held mutexes= "abort lock" "mutator lock"(shared held)
 runtime.cc:675] native: #00 pc 00000000005376cc /apex/com.android.art/lib64/libart.so (art::DumpNativeStack(std::__1::basic_ostream >&, int, BacktraceMap*, char const*, art::ArtMethod*, void*, bool)+128) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #01 pc 00000000006f0e24 /apex/com.android.art/lib64/libart.so (art::Thread::DumpStack(std::__1::basic_ostream >&, bool, BacktraceMap*, bool) const+236) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #02 pc 00000000006fe6b0 /apex/com.android.art/lib64/libart.so (art::DumpCheckpoint::Run(art::Thread*)+208) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #03 pc 0000000000364174 /apex/com.android.art/lib64/libart.so (art::ThreadList::RunCheckpoint(art::Closure*, art::Closure*)+440) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #04 pc 00000000006fce50 /apex/com.android.art/lib64/libart.so (art::ThreadList::Dump(std::__1::basic_ostream >&, bool)+280) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #05 pc 00000000006d7ec4 /apex/com.android.art/lib64/libart.so (art::AbortState::Dump(std::__1::basic_ostream >&) const+212) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #06 pc 00000000006d2b24 /apex/com.android.art/lib64/libart.so (art::Runtime::Abort(char const*)+1016) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #07 pc 0000000000016ea8 /apex/com.android.art/lib64/libbase.so (android::base::SetAborter(std::__1::function<void>&&)::$_3::__invoke(char const*)+80) (BuildId: b77c57f68a484ed93d5a7eda59d83bf9)
 runtime.cc:675] native: #08 pc 0000000000016450 /apex/com.android.art/lib64/libbase.so (android::base::LogMessage::~LogMessage()+352) (BuildId: b77c57f68a484ed93d5a7eda59d83bf9)
 runtime.cc:675] native: #09 pc 0000000000445224 /apex/com.android.art/lib64/libart.so (art::JavaVMExt::JniAbort(char const*, char const*)+1612) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #10 pc 0000000000329258 /apex/com.android.art/lib64/libart.so (art::JavaVMExt::JniAbortV(char const*, char const*, std::__va_list)+108) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #11 pc 000000000048d59c /apex/com.android.art/lib64/libart.so (art::(anonymous namespace)::ScopedCheck::AbortF(char const*, ...) (.__uniq.99033978352804627313491551960229047428)+144) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #12 pc 0000000000453ad0 /apex/com.android.art/lib64/libart.so (art::(anonymous namespace)::ScopedCheck::Check(art::ScopedObjectAccess&, bool, char const*, art::(anonymous namespace)::JniValueType*) (.__uniq.99033978352804627313491551960229047428)+4648) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #13 pc 0000000000475e2c /apex/com.android.art/lib64/libart.so (art::(anonymous namespace)::CheckJNI::CheckCallArgs(art::ScopedObjectAccess&, art::(anonymous namespace)::ScopedCheck&, _JNIEnv*, _jobject*, _jclass*, _jmethodID*, art::InvokeType, art::(anonymous namespace)::VarArgs const*) (.__uniq.99033978352804627313491551960229047428)+76) (BuildId: 56e704c544e6c624201be2ab4933e853)
A/e.androidplaye: runtime.cc:675] native: #14 pc 0000000000474d4c /apex/com.android.art/lib64/libart.so (art::(anonymous namespace)::CheckJNI::CallMethodV(char const*, _JNIEnv*, _jobject*, _jclass*, _jmethodID*, std::__va_list, art::Primitive::Type, art::InvokeType) (.__uniq.99033978352804627313491551960229047428)+268) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #15 pc 00000000005ca9ec /apex/com.android.art/lib64/libart.so (art::(anonymous namespace)::CheckJNI::CallVoidMethodV(_JNIEnv*, _jobject*, _jmethodID*, std::__va_list) (.__uniq.99033978352804627313491551960229047428.llvm.5591279935177935698)+72) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #16 pc 0000000000024ad4 /data/app/~~kWvaZj7JOt9YyFawweYvtw==/com.example.androidplayer-_Vqw2zfb57LpijMAPW6U4w==/base.apk (offset 174000) (_JNIEnv::CallVoidMethod(_jobject*, _jmethodID*, ...)+172) (BuildId: 2f8cba1fdcec13c03a15bf3de18a4ddd1c91325d)
 runtime.cc:675] native: #17 pc 000000000002173c /data/app/~~kWvaZj7JOt9YyFawweYvtw==/com.example.androidplayer-_Vqw2zfb57LpijMAPW6U4w==/base.apk (offset 174000) (ffmpeg::AndroidPlayer::handleAudioFrame(ffmpeg::Frame const&)+184) (BuildId: 2f8cba1fdcec13c03a15bf3de18a4ddd1c91325d)
 runtime.cc:675] native: #18 pc 0000000000028488 /data/app/~~kWvaZj7JOt9YyFawweYvtw==/com.example.androidplayer-_Vqw2zfb57LpijMAPW6U4w==/base.apk (offset 174000) (???) (BuildId: 2f8cba1fdcec13c03a15bf3de18a4ddd1c91325d)
 runtime.cc:675] native: #19 pc 00000000000283c0 /data/app/~~kWvaZj7JOt9YyFawweYvtw==/com.example.androidplayer-_Vqw2zfb57LpijMAPW6U4w==/base.apk (offset 174000) (???) (BuildId: 2f8cba1fdcec13c03a15bf3de18a4ddd1c91325d)
 runtime.cc:675] native: #20 pc 000000000002835c /data/app/~~kWvaZj7JOt9YyFawweYvtw==/com.example.androidplayer-_Vqw2zfb57LpijMAPW6U4w==/base.apk (offset 174000) (std::__ndk1::__bind_return<void> >, std::__ndk1::tuple, __is_valid_bind_return<void> >, std::__ndk1::tuple >::value>::type std::__ndk1::__bind<void> const&>::operator()(ffmpeg::Frame const&)+76) (BuildId: 2f8cba1fdcec13c03a15bf3de18a4ddd1c91325d)
 runtime.cc:675] native: #21 pc 00000000000282ec /data/app/~~kWvaZj7JOt9YyFawweYvtw==/com.example.androidplayer-_Vqw2zfb57LpijMAPW6U4w==/base.apk (offset 174000) (???) (BuildId: 2f8cba1fdcec13c03a15bf3de18a4ddd1c91325d)
 runtime.cc:675] native: #22 pc 0000000000028288 /data/app/~~kWvaZj7JOt9YyFawweYvtw==/com.example.androidplayer-_Vqw2zfb57LpijMAPW6U4w==/base.apk (offset 174000) (void std::__ndk1::__invoke_void_return_wrapper<void>::__call const&>&, ffmpeg::Frame const&>(std::__ndk1::__bind<void> const&>&, ffmpeg::Frame const&)+48) (BuildId: 2f8cba1fdcec13c03a15bf3de18a4ddd1c91325d)
 runtime.cc:675] native: #23 pc 0000000000028234 /data/app/~~kWvaZj7JOt9YyFawweYvtw==/com.example.androidplayer-_Vqw2zfb57LpijMAPW6U4w==/base.apk (offset 174000) (???) (BuildId: 2f8cba1fdcec13c03a15bf3de18a4ddd1c91325d)
A/e.androidplaye: runtime.cc:675] native: #24 pc 000000000002725c /data/app/~~kWvaZj7JOt9YyFawweYvtw==/com.example.androidplayer-_Vqw2zfb57LpijMAPW6U4w==/base.apk (offset 174000) (std::__ndk1::__function::__func const&>, std::__ndk1::allocator const&> >, void (ffmpeg::Frame const&)>::operator()(ffmpeg::Frame const&)+48) (BuildId: 2f8cba1fdcec13c03a15bf3de18a4ddd1c91325d)
 runtime.cc:675] native: #25 pc 000000000004039c /data/app/~~kWvaZj7JOt9YyFawweYvtw==/com.example.androidplayer-_Vqw2zfb57LpijMAPW6U4w==/base.apk (offset 380000) (???) (BuildId: 9a9d303cb440eed7364800d0ead0962a65175480)
 runtime.cc:675] native: #26 pc 000000000003b1f8 /data/app/~~kWvaZj7JOt9YyFawweYvtw==/com.example.androidplayer-_Vqw2zfb57LpijMAPW6U4w==/base.apk (offset 380000) (std::__ndk1::function<void>::operator()(ffmpeg::Frame const&) const+44) (BuildId: 9a9d303cb440eed7364800d0ead0962a65175480)
 runtime.cc:675] native: #27 pc 0000000000039dd4 /data/app/~~kWvaZj7JOt9YyFawweYvtw==/com.example.androidplayer-_Vqw2zfb57LpijMAPW6U4w==/base.apk (offset 380000) (ffmpeg::Player::playAudio()+164) (BuildId: 9a9d303cb440eed7364800d0ead0962a65175480)
 runtime.cc:675] native: #28 pc 000000000003db90 /data/app/~~kWvaZj7JOt9YyFawweYvtw==/com.example.androidplayer-_Vqw2zfb57LpijMAPW6U4w==/base.apk (offset 380000) (???) (BuildId: 9a9d303cb440eed7364800d0ead0962a65175480)
 runtime.cc:675] native: #29 pc 000000000003db1c /data/app/~~kWvaZj7JOt9YyFawweYvtw==/com.example.androidplayer-_Vqw2zfb57LpijMAPW6U4w==/base.apk (offset 380000) (???) (BuildId: 9a9d303cb440eed7364800d0ead0962a65175480)
 runtime.cc:675] native: #30 pc 000000000003dacc /data/app/~~kWvaZj7JOt9YyFawweYvtw==/com.example.androidplayer-_Vqw2zfb57LpijMAPW6U4w==/base.apk (offset 380000) (???) (BuildId: 9a9d303cb440eed7364800d0ead0962a65175480)
 runtime.cc:675] native: #31 pc 000000000003da10 /data/app/~~kWvaZj7JOt9YyFawweYvtw==/com.example.androidplayer-_Vqw2zfb57LpijMAPW6U4w==/base.apk (offset 380000) (???) (BuildId: 9a9d303cb440eed7364800d0ead0962a65175480)
 runtime.cc:675] native: #32 pc 000000000003e6cc /data/app/~~kWvaZj7JOt9YyFawweYvtw==/com.example.androidplayer-_Vqw2zfb57LpijMAPW6U4w==/base.apk (offset 380000) (???) (BuildId: 9a9d303cb440eed7364800d0ead0962a65175480)
 runtime.cc:675] native: #33 pc 000000000003e5e0 /data/app/~~kWvaZj7JOt9YyFawweYvtw==/com.example.androidplayer-_Vqw2zfb57LpijMAPW6U4w==/base.apk (offset 380000) (???) (BuildId: 9a9d303cb440eed7364800d0ead0962a65175480)
 runtime.cc:675] native: #34 pc 000000000003df50 /data/app/~~kWvaZj7JOt9YyFawweYvtw==/com.example.androidplayer-_Vqw2zfb57LpijMAPW6U4w==/base.apk (offset 380000) (???) (BuildId: 9a9d303cb440eed7364800d0ead0962a65175480)
 runtime.cc:675] native: #35 pc 00000000000efb14 /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+264) (BuildId: 94065bf91428f6ae9fb310c478171302)
 runtime.cc:675] native: #36 pc 000000000008c35c /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+68) (BuildId: 94065bf91428f6ae9fb310c478171302)
 runtime.cc:675] (no managed stack frames)
 runtime.cc:675] 
 runtime.cc:675] "main" prio=10 tid=1 Native
 runtime.cc:675] | group="" sCount=1 ucsCount=0 flags=1 obj=0x71915d18 self=0xb400007bd4472800
 runtime.cc:675] | sysTid=3241 nice=-10 cgrp=default sched=0/0 handle=0x7bd5ac64f8
 runtime.cc:675] | state=S schedstat=( 20358355629 2398850818 13099 ) utm=1560 stm=474 core=5 HZ=100
 runtime.cc:675] | stack=0x7ff2616000-0x7ff2618000 stackSize=8188KB
 runtime.cc:675] | held mutexes=
 runtime.cc:675] native: #00 pc 0000000000086f90 /apex/com.android.runtime/lib64/bionic/libc.so (syscall+32) (BuildId: 94065bf91428f6ae9fb310c478171302)
A/e.androidplaye: runtime.cc:675] native: #01 pc 000000000047cc80 /apex/com.android.art/lib64/libart.so (art::ConditionVariable::WaitHoldingLocks(art::Thread*)+140) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #02 pc 00000000005c8cac /apex/com.android.art/lib64/libart.so (art::(anonymous namespace)::CheckJNI::FindClass(_JNIEnv*, char const*) (.__uniq.99033978352804627313491551960229047428.llvm.5591279935177935698)+1096) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #03 pc 000000000011aae0 /system/lib64/libandroid_runtime.so (android::NativeDisplayEventReceiver::dispatchVsync(long, android::PhysicalDisplayId, unsigned int, android::VsyncEventData)+80) (BuildId: f73b7e7fab15dc5b96680181e111fb8f)
 runtime.cc:675] native: #04 pc 00000000000af3f0 /system/lib64/libgui.so (android::DisplayEventDispatcher::handleEvent(int, int, void*)+204) (BuildId: 3f01a4a30b5fa3a4804b8361ddccc4a3)
 runtime.cc:675] native: #05 pc 0000000000018184 /system/lib64/libutils.so (android::Looper::pollInner(int)+916) (BuildId: 16796d84bdcf185b2112267dbd820c19)
 runtime.cc:675] native: #06 pc 0000000000017d84 /system/lib64/libutils.so (android::Looper::pollOnce(int, int*, int*, void**)+116) (BuildId: 16796d84bdcf185b2112267dbd820c19)
 runtime.cc:675] native: #07 pc 0000000000154668 /system/lib64/libandroid_runtime.so (android::android_os_MessageQueue_nativePollOnce(_JNIEnv*, _jobject*, long, int)+48) (BuildId: f73b7e7fab15dc5b96680181e111fb8f)
 runtime.cc:675] at android.os.MessageQueue.nativePollOnce(Native method)
 runtime.cc:675] at android.os.MessageQueue.next(MessageQueue.java:337)
 runtime.cc:675] at android.os.Looper.loopOnce(Looper.java:168)
 runtime.cc:675] at android.os.Looper.loop(Looper.java:299)
 runtime.cc:675] at android.app.ActivityThread.main(ActivityThread.java:8250)
 runtime.cc:675] at java.lang.reflect.Method.invoke(Native method)
 runtime.cc:675] at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:556)
 runtime.cc:675] at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1045)
 runtime.cc:675] 
 runtime.cc:675] "Signal Catcher" prio=10 tid=4 WaitingInMainSignalCatcherLoop
 runtime.cc:675] | group="" sCount=1 ucsCount=0 flags=1 obj=0x13340270 self=0xb400007b1f611800
 runtime.cc:675] | sysTid=3247 nice=-20 cgrp=default sched=0/0 handle=0x7b209aecb0
 runtime.cc:675] | state=S schedstat=( 300521 0 2 ) utm=0 stm=0 core=6 HZ=100
 runtime.cc:675] | stack=0x7b208b7000-0x7b208b9000 stackSize=991KB
 runtime.cc:675] | held mutexes=
 runtime.cc:675] native: #00 pc 00000000000db77c /apex/com.android.runtime/lib64/bionic/libc.so (__rt_sigtimedwait+12) (BuildId: 94065bf91428f6ae9fb310c478171302)
 runtime.cc:675] native: #01 pc 000000000009a1b8 /apex/com.android.runtime/lib64/bionic/libc.so (sigwait64+92) (BuildId: 94065bf91428f6ae9fb310c478171302)
 runtime.cc:675] native: #02 pc 000000000057420c /apex/com.android.art/lib64/libart.so (art::SignalCatcher::WaitForSignal(art::Thread*, art::SignalSet&)+108) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #03 pc 0000000000573e1c /apex/com.android.art/lib64/libart.so (art::SignalCatcher::Run(void*)+228) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #04 pc 00000000000efb14 /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+264) (BuildId: 94065bf91428f6ae9fb310c478171302)
 runtime.cc:675] native: #05 pc 000000000008c35c /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+68) (BuildId: 94065bf91428f6ae9fb310c478171302)
 runtime.cc:675] (no managed stack frames)
 runtime.cc:675] 
 runtime.cc:675] "perfetto_hprof_listener" prio=10 tid=7 Native (still starting up)
 runtime.cc:675] | group="" sCount=1 ucsCount=0 flags=1 obj=0x0 self=0xb400007b138a8800
 runtime.cc:675] | sysTid=3248 nice=-20 cgrp=default sched=0/0 handle=0x7b208b0cb0
 runtime.cc:675] | state=S schedstat=( 175052 0 3 ) utm=0 stm=0 core=6 HZ=100
A/e.androidplaye: runtime.cc:675] | stack=0x7b207b9000-0x7b207bb000 stackSize=991KB
 runtime.cc:675] | held mutexes=
 runtime.cc:675] native: #00 pc 00000000000daad8 /apex/com.android.runtime/lib64/bionic/libc.so (read+8) (BuildId: 94065bf91428f6ae9fb310c478171302)
 runtime.cc:675] native: #01 pc 000000000001d840 /apex/com.android.art/lib64/libperfetto_hprof.so (void* std::__1::__thread_proxy >, ArtPlugin_Initialize::$_34> >(void*)+260) (BuildId: d60314cb99f035c98d20ab286d52fc6d)
 runtime.cc:675] native: #02 pc 00000000000efb14 /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+264) (BuildId: 94065bf91428f6ae9fb310c478171302)
 runtime.cc:675] native: #03 pc 000000000008c35c /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+68) (BuildId: 94065bf91428f6ae9fb310c478171302)
 runtime.cc:675] (no managed stack frames)
 runtime.cc:675] 
 runtime.cc:675] "ADB-JDWP Connection Control Thread" prio=10 tid=8 WaitingInMainDebuggerLoop
 runtime.cc:675] | group="" sCount=1 ucsCount=0 flags=1 obj=0x133402e8 self=0xb400007b1f64d800
 runtime.cc:675] | sysTid=3249 nice=-20 cgrp=default sched=0/0 handle=0x7b207b2cb0
 runtime.cc:675] | state=S schedstat=( 935055 0 13 ) utm=0 stm=0 core=4 HZ=100
 runtime.cc:675] | stack=0x7b206bb000-0x7b206bd000 stackSize=991KB
 runtime.cc:675] | held mutexes=
 runtime.cc:675] native: #00 pc 00000000000dbe3c /apex/com.android.runtime/lib64/bionic/libc.so (__ppoll+12) (BuildId: 94065bf91428f6ae9fb310c478171302)
 runtime.cc:675] native: #01 pc 00000000000973b4 /apex/com.android.runtime/lib64/bionic/libc.so (poll+96) (BuildId: 94065bf91428f6ae9fb310c478171302)
 runtime.cc:675] native: #02 pc 00000000000099e4 /apex/com.android.art/lib64/libadbconnection.so (adbconnection::AdbConnectionState::RunPollLoop(art::Thread*)+724) (BuildId: c4c03b03ed50414e876ae23cd04eae7e)
 runtime.cc:675] native: #03 pc 00000000000080ac /apex/com.android.art/lib64/libadbconnection.so (adbconnection::CallbackFunction(void*)+1320) (BuildId: c4c03b03ed50414e876ae23cd04eae7e)
 runtime.cc:675] native: #04 pc 00000000000efb14 /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+264) (BuildId: 94065bf91428f6ae9fb310c478171302)
 runtime.cc:675] native: #05 pc 000000000008c35c /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+68) (BuildId: 94065bf91428f6ae9fb310c478171302)
 runtime.cc:675] (no managed stack frames)
 runtime.cc:675] 
 runtime.cc:675] "Jit thread pool worker thread 0" prio=5 tid=9 Native
 runtime.cc:675] | group="" sCount=1 ucsCount=0 flags=1 obj=0x13340360 self=0xb400007b138c4000
 runtime.cc:675] | sysTid=3250 nice=9 cgrp=default sched=0/0 handle=0x7b206b4cb0
 runtime.cc:675] | state=S schedstat=( 1384418878 710723029 1368 ) utm=118 stm=20 core=1 HZ=100
 runtime.cc:675] | stack=0x7b205b5000-0x7b205b7000 stackSize=1023KB
 runtime.cc:675] | held mutexes=
 runtime.cc:675] native: #00 pc 0000000000086f90 /apex/com.android.runtime/lib64/bionic/libc.so (syscall+32) (BuildId: 94065bf91428f6ae9fb310c478171302)
 runtime.cc:675] native: #01 pc 000000000047cc80 /apex/com.android.art/lib64/libart.so (art::ConditionVariable::WaitHoldingLocks(art::Thread*)+140) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #02 pc 000000000047cb18 /apex/com.android.art/lib64/libart.so (art::ThreadPool::GetTask(art::Thread*)+120) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #03 pc 0000000000619810 /apex/com.android.art/lib64/libart.so (art::ThreadPoolWorker::Run()+136) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #04 pc 00000000006196f0 /apex/com.android.art/lib64/libart.so (art::ThreadPoolWorker::Callback(void*)+160) (BuildId: 56e704c544e6c624201be2ab4933e853)
A/e.androidplaye: runtime.cc:675] native: #05 pc 00000000000efb14 /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+264) (BuildId: 94065bf91428f6ae9fb310c478171302)
 runtime.cc:675] native: #06 pc 000000000008c35c /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+68) (BuildId: 94065bf91428f6ae9fb310c478171302)
 runtime.cc:675] (no managed stack frames)
 runtime.cc:675] 
 runtime.cc:675] "HeapTaskDaemon" prio=5 tid=10 Native
 runtime.cc:675] | group="" sCount=1 ucsCount=0 flags=1 obj=0x13340d60 self=0xb400007b138d5800
 runtime.cc:675] | sysTid=3256 nice=4 cgrp=default sched=0/0 handle=0x7b205aecb0
 runtime.cc:675] | state=S schedstat=( 12936736517 2503386925 3722 ) utm=1074 stm=219 core=4 HZ=100
 runtime.cc:675] | stack=0x7b204ab000-0x7b204ad000 stackSize=1039KB
 runtime.cc:675] | held mutexes=
 runtime.cc:675] native: #00 pc 0000000000086f90 /apex/com.android.runtime/lib64/bionic/libc.so (syscall+32) (BuildId: 94065bf91428f6ae9fb310c478171302)
 runtime.cc:675] native: #01 pc 000000000047cc80 /apex/com.android.art/lib64/libart.so (art::ConditionVariable::WaitHoldingLocks(art::Thread*)+140) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #02 pc 000000000056bbb8 /apex/com.android.art/lib64/libart.so (art::gc::Heap::TrimIndirectReferenceTables(art::Thread*)+1364) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #03 pc 000000000056b468 /apex/com.android.art/lib64/libart.so (art::gc::Heap::Trim(art::Thread*)+68) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #04 pc 000000000056b3cc /apex/com.android.art/lib64/libart.so (art::gc::Heap::HeapTrimTask::Run(art::Thread*)+32) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #05 pc 000000000046ce28 /apex/com.android.art/lib64/libart.so (art::gc::TaskProcessor::RunAllTasks(art::Thread*)+56) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] at dalvik.system.VMRuntime.runHeapTasks(Native method)
 runtime.cc:675] at java.lang.Daemons$HeapTaskDaemon.runInternal(Daemons.java:609)
 runtime.cc:675] at java.lang.Daemons$Daemon.run(Daemons.java:140)
 runtime.cc:675] at java.lang.Thread.run(Thread.java:1012)
 runtime.cc:675] 
 runtime.cc:675] "ReferenceQueueDaemon" prio=5 tid=11 Waiting
 runtime.cc:675] | group="" sCount=1 ucsCount=0 flags=1 obj=0x133403d8 self=0xb400007b138d7400
 runtime.cc:675] | sysTid=3259 nice=4 cgrp=default sched=0/0 handle=0x7b204a4cb0
 runtime.cc:675] | state=S schedstat=( 2230226662 207087138 1228 ) utm=26 stm=196 core=4 HZ=100
 runtime.cc:675] | stack=0x7b203a1000-0x7b203a3000 stackSize=1039KB
 runtime.cc:675] | held mutexes=
 runtime.cc:675] native: #00 pc 0000000000086f90 /apex/com.android.runtime/lib64/bionic/libc.so (syscall+32) (BuildId: 94065bf91428f6ae9fb310c478171302)
 runtime.cc:675] native: #01 pc 000000000047cc80 /apex/com.android.art/lib64/libart.so (art::ConditionVariable::WaitHoldingLocks(art::Thread*)+140) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] native: #02 pc 0000000000486588 /apex/com.android.art/lib64/libart.so (art::Monitor::Wait(art::Thread*, art::ObjPtr, long, int, bool, art::ThreadState)+2520) (BuildId: 56e704c544e6c624201be2ab4933e853)
 runtime.cc:675] at java.lang.Object.wait(Native method)
 runtime.cc:675] - waiting on <0x0483c441> (a java.lang.Class)
 runtime.cc:675] at java.lang.Object.wait(Object.java:442)
 runtime.cc:675] at java.lang.Object.wait(Object.java:568)
 runtime.cc:675] at java.lang.Daemons$ReferenceQueueDaemon.runInternal(Daemons.java:232)
 runtime.cc:675] - locked <0x0483c441> (a java.lang.Class)
 runtime.cc:675] at java.lang.Daemons$Daemon.run(Daemons.java:140)
 runtime.cc:675] at java.lang.Thread.run(Thread.java:1012)
 runtime.cc:675] 
...(this part i missed because so big)
W/e.androidplaye: Suspending all threads took: 10.935ms
A/e.androidplaye: runtime.cc:683] JNI DETECTED ERROR IN APPLICATION: thread Thread[2,tid=3420,Native,Thread*=0xb400007bd4470c00,peer=0x12cc1510,"Thread-8687"] using JNIEnv* from thread Thread[3,tid=3419,Native,Thread*=0xb400007aadc2e400,peer=0x12d00000,"Thread-8688"]
 runtime.cc:683] in call to CallVoidMethodV
A/libc: Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 3420 (Thread-8687), pid 3241 (e.androidplayer)
</void></void></void></void></void></void></void>

-
ISO-9660 Compromise, Part 2 : Finding Root
25 octobre 2021, par Multimedia Mike — GeneralA long time ago, I dashed off a quick blog post with a curious finding after studying the ISO-9660 spec : The format stores multi-byte numbers in a format I termed “omni-endian”– the committee developing the format apparently couldn’t come to an agreement on this basic point regarding big- vs. little-endian encoding (I’m envisioning something along the lines of “tastes great ! … less filling !” in the committee meetings).
I recently discovered another bit of compromise in the ISO-9660 spec : It seems that there are 2 different methods for processing the directory structure. That means it’s incumbent upon ISO-9660 creation software to fill in the data structures to support both methods, because some ISO-reading programs out there rely on one set of data structures while the rest prefer to read the other set.
Background
As a refresher, the “ISO” extension of an ISO file refers to the ISO-9660 specification. This is a type of read-only filesystem (i.e, the filesystem is created once and never updated after initial creation) for the purpose of storing on a read-only medium, often an optical disc (CD-ROM, DVD-ROM). The level of nostalgic interest I display for the ISO-9660 filesystem reminds me of my computer science curriculum professors from the mid-90s reminiscing about ye olden days of punchcard programming, but such is my lot. I’m probably also alone in my frustration of seeing rips of, e.g., GameCube or Xbox or 3DO games being tagged with the extension .ISO since those systems use different read-only filesystems.
I recently fell in with an odd bunch called the eXoDOS project and was trying to help fill in a few gaps. One request was a 1994 game called Power Drive for DOS.
My usual CD-ROM ripping method (for the data track) is a simple ‘dd’ command from a Linux command line to copy the string of raw sectors. However, it turned out to be unusually difficult to open the resulting ISO. A few of the the options I know of worked but most didn’t. What’s the difference ?
Methods that work :
- Mounting the file with the Linux iso9660 kernel module, i.e.,
mount -t iso9660 /dev/optical-drive /mnt
or
mount -t iso9660 -o loop /path/to/Power-Drive.iso /mnt
- Directory Opus
- Windows 10 can read the filesystem when reading the physical disc
- Windows 10 can burn the ISO image to a new CD (“right click” -> “Burn disc image”) ; this method does not modify any of the existing sectors but did append 149 additional empty sectors
Methods that don’t work :
- fuseiso
- Dosbox
- Winrar
- 7zip
- Daemon Tools
- Imgburn
- Internet Archive’s ISO lister (“View contents” on the ISO file)
Understanding The Difference
I think I might have a handle on why some tools are able to process this disc while most can’t. There appears to be 2 sets of data structures to describe the base of the filesystem : A root directory, and a path table. These both occur in the first substantive sector of the ISO-9660 filesystem, usually sector 16.
A compact disc can be abstractly visualized as a long string of sectors, each one 2,352 bytes long. (See my Grand Unified Theory of Compact Disc post for deeper discussion.) A CD-ROM data track will contain 2048 bytes of data. Thus, sector 16 appears at 0x8000 of an ISO filesystem. I like the clarity of this description of the ISO-9660 spec. It shows that the path table is defined at byte 140 (little-endian ; big comes later) and location of the root directory is at byte 158. Thus, these locations generally occur at 0x808c and 0x809e.
Primary Volume Descriptor
The path table is highlighted in green and the root directory record is highlighted in red. These absolute locations are specified in sectors. So the path table is located at sector 0x12 = offset 0x9000 in the image, while the root directory record is supposed to be at sector 0x62 = 0x31000. Checking into those sectors, it turns out that the path table is valid while the root directory record is invalid. Thus, any tool that relies on the path table will be successful in interpreting the disc, while tools that attempt to recursively traverse starting from root directory record are gonna have a bad time.
Since I was able to view the filesystem with a few different tools, I know what the root directory contains. Searching for those filenames reveals that the root directory was supposed to point to the next sector, number 0x63. So this was a bizarre off-by-1 error on the part of the ISO creation tool. Maybe. I manually corrected 0x62 -> 0x63 and that fixed the interaction with fuseiso, but not with other tools. So there may have been some other errors. Note that a quick spot-check of another, functional ISO revealed that this root directory sector is supposed to be exact, not 1-indexed.
Upon further inspection, I noticed that, while fuseiso appeared to work with that one patch, none of the files returned correct data, and none of the directories contained anything. That’s when I noticed that ALL of the sector locations described in the various directory and file records are off by 1 !
Further Investigation
I have occasionally run across ISO images on the Internet Archive that return the error about not being able to read the contents when trying to “View contents” (error text : “failed to obtain file list from xyz.iso”, as seen with this ISO). Too bad I didn’t make a record of them because I would be interested to see if they have the same corruption.
Eventually, I’ll probably be able to compile an archive of deviant ISO-9660 images. A few months ago, I was processing a large collection from IA and found a corrupted ISO which had a cycle, i.e., the subdirectory pointed to a parent directory, which caused various ISO tools to loop forever. Just one of those things that is “never supposed to happen”, so why write code to deal with it gracefully ?
See Also
The post ISO-9660 Compromise, Part 2 : Finding Root first appeared on Breaking Eggs And Making Omelettes.
- Mounting the file with the Linux iso9660 kernel module, i.e.,
Syndiquer tout le site
