Recherche avancée

Sur d’autres sites (7357)

• Reverse Engineering Radius VideoVision

  3 avril 2011, par Multimedia Mike — Reverse Engineering

  I was called upon to help reverse engineer an old video codec called VideoVision (FourCC : PGVV), ostensibly from a company named Radius. I’m not sure of the details exactly but I think a game developer has a bunch of original FMV data from an old game locked up in this format. The name of the codec sounded familiar. Indeed, we have had a sample in the repository since 2002. Alex B. did some wiki work on the codec some years ago. The wiki mentions that there existed a tool to transcode PGVV data into MJPEG-B data, which is already known and supported by FFmpeg.

  The Software
  My contacts were able to point me to some software, now safely archived in the PGVV samples directory. There is StudioPlayer2.6.2.sit.hqx which is supposed to be a QuickTime component for working with PGVV data. I can’t even remember how to deal with .sit or .hqx data. Then there is RadiusVVTranscoder101.zip which is the tool that transcodes to MJPEG-B.

  Disassembling for Reverse Engineering
  Since I could actually unpack the transcoder, I set my sights on that. Unpacking the archive sets up a directory structure for a component. There is a binary called RadiusVVTranscoder under RadiusVVTranscoder.component/Contents/MacOS/. Basic deadlisting disassembly is performed via ’otool’ as shown :

    otool -tV RadiusVVTranscoder | c++filt
  

  This results in a deadlisting of both PowerPC and 32-bit x86 code, as the binary is a "fat" Mac OS X binary designed to run on both architectures. The command line also demangles C++ function signatures which gives useful insight into the parameters passed to a function.

  Pretty Pictures
  The binary had a lot of descriptive symbols. As a basis for reverse engineering, I constructed call graphs using these symbols. Here are the 2 most relevant portions (click for larger images).

  The codec initialization generates Huffman tables relevant to the codec :

  
  
  

  The main decode function calls AddMJPGFrame which apparently does the heavy lifting for the transcode process :

  
  
  

  Based on this tree, I’m guessing that luma blocks can be losslessly transcoded (perhaps with different Huffman tables) which chroma blocks may rely on a different quantization method.

  Assembly Constructs
  I started looking at the instructions (the x86 ones, of course). The binary uses a calling convention I haven’t seen before, at least not for the x86 : Rather than pushing function arguments onto the stack, the code manually subtracts, e.g., 12 from the ESP register, loads 3 32-bit arguments into memory relative to ESP, and then proceeds with the function call.

  I’m also a little unclear on constructs such as "call ___i686.get_pc_thunk.bx" seen throughout relevant functions such as MakeRadiusQuantizationTables().

  I’m just presenting what I have so far in case anyone else wants to try their hand.

• How Funnel for Piwik Analytics enriches your Piwik experience giving you ultimate insights and debugging capabilities

  13 janvier 2017, par InnoCraft — Community

  No matter what type of website or app you have, whether you are trying to get your users to sign up for something or sell products, there is a certain number of steps your visitors have to go through. On every step you lose visitors and therefore potential revenue and conversions. Therefore it is critical to know where your visitors actually follow those steps in your website or app, where you lose them and where your visitors maybe get confused. By defining a funnel, you can improve your conversion rates, sales and revenue as you can exactly determine where you lose your visitors in converting your goal or a sale.

  A Funnel defines a series of steps that you expect your visitors to take on their way to converting a goal. Funnels, a premium feature for Piwik developed by InnoCraft, lets you create funnels to get the data you need to improve your websites and mobile apps. Learn more about Funnel.

  In this blog post we will cover the reports the Funnel plugin provides. The next blog post shows you how to configure and validate your funnel in Piwik.

  

  Integration in Goal reports

  At Piwik and InnoCraft, we usually start looking into our goal reports. Funnel integrates directly into each goal reporting page giving you a quick overview how your funnel is doing. This saves us a lot of time as we don’t have to separately look into each funnel page and only takes us maybe an additional second to keep an eye on our funnels. By clicking on the headline or “View funnel report” link, you can directly go to the funnel report to get a more detailed report if you notice any spike in the evolution of the conversions or conversion rate.

  

  Getting an overall Funnel overview

  Next we usually go to the “Funnel Overview” page where it shows a list of all activated Funnels and their performance over time. You will find the look familiar as it is similar to the “Goals Overview” page. If we find something unusual there, for example any spikes, we usually directly click on the headline of the Funnel to go to the detailed Funnel report. You can also choose a funnel from the left reporting menu or search for a funnel by entering the shortcut “f”.

  

  Viewing a funnel report

  A funnel reporting page looks very similar to a Goal reporting page. It starts with an evolution graph and sparklines showing you the performance of your funnel over time.

  

  In the evolution graph you can select the metrics you want to plot. We usually have an eye on the funnel conversion rate and the number of “Funnel entries” or the number of “Funnel conversions”. The conversion rate alone does not show you how your funnel is performing. Imagine the rate is always stable at around 20% and you might think everything is alright, but if the number of visitors that take part in your funnel goes down, you might have a problem as the number of funnel conversions actually decreases even though the rate is the same. So we recommend to not only have a look at the conversion rate. The report will remember the metrics you want to plot each time you open it so you don’t have to re-select them over and over again.

  

  The funnel overview

  In the funnel overview we are giving you more details about the funnel and goal related conversion metrics so you don’t have to switch between the goal and funnel report and compare them easily.

  When you analyze a funnel report, you might not always remember how the funnel is configured. Even though you specify names for each step you sometimes need to know on which pages a certain step will be activated. By clicking on the funnel summary link you can quickly look into the funnel configuration and also see all important metrics at a glance in a simple table without having to scroll.

  

  You might also notice the Visitor Log link which will show you all actions for all visitors that have entered this funnel. This lets you really understand how your visitors navigate through your website and how they proceeded, exited or converted your funnel on a visitor level.

  

  The Funnel visualization

  Below the funnel overview you can visually see where your visitors entered, proceeded, converted and exited your funnel. We kept the UI clean so you can focus on the important things.

  Most tools only give you the pages where visitors have entered your funnel but we do better and also show you the list of external referrers used by visitors to enter your funnel directly (marketing campaigns, search engines or other websites). Also we do not only show only the top 5 pages but up to 100 pages and 50 referrers (more can be configured if needed). When you hover a row, you will not only see the number of hits but also the percentage each row has contributed to the entries. Here you want to look and understand how your visitors enter your funnel and based on the data maybe invest in successful referrers, campaigns and pages. If the pages or referrers you expect to see there don’t show up, your users might not understand the path you had in mind for them.

  

  Next you may notice how many visits have gone through each step, in this case 3487 visits. The green and red bar lets you quickly identify how many of your visitors have proceeded to the next step (green) compared to how many have exited the funnel at this step (red). Ideally, most of the bar is green and not red indicating that more visitors proceed to the next step than they exit.

  

  Now the next feature is really valuable. When you hover the step title or the number of visits, you will notice that two icons appear :

  

  Those two little icons are really powerful and give you even more insights to really dig into all the data. The left icon shows you the visitor log showing all actions of each visitor that have participated in this particular funnel step. This means for each step you get to see all the details and actions of each visitor. This lets you really debug and understand problems in your funnel.

  At InnoCraft, we understand that plain numbers are often not so valuable. Only the evolution over time, when you put the numbers in relation to something else you can really understand how your website is doing. The icon to the right lets you do exactly this, it lets you view the row evolution for each funnel step. We are sure you will enjoy this feature. It lets you explore how each funnel step is doing over time. For example the number of entries for a step or how many proceeded to the next step from here over time. Here you ideally want to see that the “Proceeded Rate” increases over time, meaning more and more visitors actually proceed to the next step instead of exiting it.

  

  We are sure you will really love those features that give you just those extra insights that other tools don’t give you.

  On the right you can find out where your visitors went to, if they did not proceed any further in the funnel. This lets you better understand why they left the funnel and did not proceed any further.

  

  At the end of the funnel report you find again the number of conversions and the conversion rate. Here we recommend looking into the visitor log when you hover the name of the last step as you can analyze how each visitor converted this funnel in detail.

  

  Applying segments

  Funnels lets you apply any Piwik segment to the Funnel report allowing you to dice your visitors multiplying the value you get out of Funnel. For example you may want to apply a segment and analyze the funnel for visitors that have visited your website or mobile app for the first time vs. recurring visitors. Sometimes it may be interesting how visitors from different countries go through your funnel, the possibilities are endless. We really recommend to take advantage of segments to understand your different target groups even better.

  The plugin also adds some new segments to your Piwik letting you segment any Piwik report by visitors that have participated in a funnel or participated in a particular funnel step. For example you could go to the “Visitors => Locations” report and apply a segment for your funnel to see which countries have participated or converted most in your funnel.

  Widgets, Scheduled Reports, and more.

  This is not where the fun ends. Funnels defines new widgets that you can add to your dashboard or export it into a third party website. You can set up scheduled reports to receive the Funnel report automatically via email or sms or download the report to share it with your colleagues. It works also very well with Custom Alerts and you can view the Funnel report in the Piwik Mobile app. You can manage Funnels via HTTP API and also fetch all Funnel reports via the HTTP Reporting API. The plugin is really nicely integrated into Piwik we will need some more blog posts to show you all the ways Funnels advances your Piwik experience and how it lets you dig into all the data so you can increase your conversions and sales based on this data.

  How to get Funnels and related features

  You can get Funnels on the Piwik Marketplace. If you want to learn more about Funnels you might be also interested in the Funnel User Guide and the Funnel FAQ.

  Similar to Funnels we also offer Users Flow which lets you visualize the flow of your users and visitors across several interactions.

• Thread in Thread

  28 mai 2014, par user3679963

  My company is working for visual effects and we set up an internal shot playback via a browser for our clients. For that we need to upload the video file to a FTP server.

  I want to convert a image sequence to mp4 and upload this file directly after the rendering will finish.

  For that I use :

  • one command prompt to convert
  • one command prompt to get an `md5hash
  • one for uploading the file

  I already achieved that on my local computer, where I just chained os.system('command').

  After recognizing that the program freezes very long with longer image sequences I changed the script to spawn a thread using the os.system chain.
  But on the Render Farm Server this script does not actually work.

  The RenderFarm Server runs Python 2.5

  There are some code examples :

  class CopraUpload(threading.Thread):
  
  
  
      # initializing Thread
  
      # via super constructor
  
      def __init__(self):
  
          threading.Thread.__init__(self)
  
  
  
  
  
      # return the size of the
  
      # created mp4 file
  
      #
  
      # @return: the file size in byte
  
      def _getFileSize(self):
  
  
  
  
  
      # creates a random id for organising
  
      # the server upload used as flag
  
      #
  
      # @return: a hash
  
      def _getHash(self):
  
              self.fileLoc = str(self.outputfileName + '.mp4')
  
              self.fileLoc = os.path.normpath(self.fileLoc)
  
              return str(os.path.getsize(self.fileLoc))
  
  
  
      # integrates the "missing" data for the xml file
  
      # generated post render from the mp4 file
  
      def _setPreviewDataToXML(self):
  
          self.xmlFile = str(self.outputfileName + '_copraUpload.xml')
  
          self.xmlFile = os.path.normpath(self.xmlFile)
  
  
  
          ett = ET.parse(self.xmlFile)
  
          root = ett.getroot()
  
          for child in root.getiterator('preview_size'):
  
              child.text = self._getFileSize()
  
          for child in root.getiterator('preview_md5hash'):
  
              child.text = self._getHash()
  
          ett.write(self.xmlFile)
  
  
  
  
  
      # create a connection to a ftp server
  
      # and copies the mp4 file and the xml file
  
      # on the server
  
      def _uploadToCopra(self):
  
  
  
          os.system(self.uploadCommand)
  
          #process = Popen(self.uploadCommand)
  
  
  
  
  
      # the main function of the program
  
      # called via start from a Thread Object
  
      def run(self):
  
  
  
          # the command which will be send to the commando shell
  
          # for further adjustments see ffmpeg help with ffmpeg.exe -h
  
          FinalCommand = self.ffmpegLocation + " -r "+ self.framerate + " -i " + self.inputPath + " -an -strict experimental -s hd720 -vcodec libx264 -preset slow -profile:v baseline -level 31 -refs 1 -maxrate 6M -bufsize 10M -vb 6M -threads 0 -g 8 -r " + self.framerate + " " + self.outputfileName + ".mp4 -y"
  
          FinalCommandList = FinalCommand.split(" ")
  
  
  
  
  
          # calling the program
  
          print "Start ffmpeg convertion"
  
  
  
          outInfo = os.path.normpath("C:\\Users\\sarender\\Desktop\\stdoutFFMPEG.txt")
  
          outError = os.path.normpath("C:\\Users\\sarender\\Desktop\\stderrFFMPEG.txt")
  
          stdoutFile = open(outInfo,"w")
  
          stderrFile = open(outError,"w")
  
  
  
          handle = subp.check_all(FinalCommandList,stdout = stdoutFile,stderr = stderrFile)
  
          handle.communicate()
  
          stdoutFile.close()
  
          stderrFile.close()
  
          print "Convertion from ffmpeg done"
  
  
  
          # fill the xml file with the missing data
  
          # - preview file size
  
          # - preview md5hash
  
          self._setPreviewDataToXML()
  
          self._uploadToCopra()
  
          print "---------------------------------->FINISHED------------------------------------------------------>"
  
  
  
  
  
      # Creates a callable Thread for the Copra Upload.
  
      # start is calling the run method which will start the Uploading

  and the main start :

      if "$(RenderSet.writenode)" == "PREVIEW":
  
          print "---------------------------------->Initializing Script------------------------------------------------------>"
  
          process = CopraUpload()
  
          process.start()

  What happens :

  The script starts after the rendering and ffmpeg converts the image sequence and creates an mp4. But it stops after that. It does not print "Conversion from ffmpeg complet". Just stops the script.

  It actually should create the Thread converting with ffmpeg and wait until it finishes. After it should write some stuff in an xml file and upload both to the server.

  Do I miss something ? Is subprocess within a thread not the way to go ? But I need a Thread because I can not deadlock the render management server.