java - Choppy audio when decoding audio/video with xuggler -

so i'm writing audio decoder pre-existing video decoder works in libgdx. problem is, when audio code wasn't threaded, audio , video choppy. audio play chunk, , video play chunk.

my solution multithreading, , let video stuff it's work (because libgdx render threads not threadsafe, , messing them causes bad things without fail). natural choice use threading stuff audio.

this fixes video choppiness, not audio still choppy, it's got artifacts on place.

this first ever stab @ serious audio programming, keep in mind might not know basic.the executor service singlethreadexecutor, idea audio need decoded , written out in-order.

here's update method:

public boolean update(float dtseconds) {     if(playstate != playstate.playing) return false;      long dtmilliseconds = (long)(dtseconds * 1000);     playtimemilliseconds += dtmilliseconds;      sleeptimeoutmilliseconds = (long) math.max(0, sleeptimeoutmilliseconds - dtmilliseconds);     if(sleeptimeoutmilliseconds > 0) {         // playhead still ahead of current frame - nothing         return false;     }       while(true) {         int packet_read_result = container.readnextpacket(packet);          if(packet_read_result < 0) {             // got bad packet - we've reached end of video stream             stop();             return true;         }           if(packet.getstreamindex() == videostreamid)          {             // have valid packet our stream              // allocate new picture data out of xuggler             ivideopicture picture = ivideopicture.make(                 videocoder.getpixeltype(),                 videocoder.getwidth(),                 videocoder.getheight()             );              // attempt read entire packet             int offset = 0;             while(offset < packet.getsize()) {                 // decode video, checking errors                 int bytesdecoded = videocoder.decodevideo(picture, packet, offset);                 if (bytesdecoded < 0) {                     throw new runtimeexception("got error decoding video");                 }                 offset += bytesdecoded;                  /* decoders consume data in packet, not                  * able construct full video picture yet. therefore                  * should check if got complete picture                  * decoder                  */                 if (picture.iscomplete()) {                     // we've read entire packet                     ivideopicture newpic = picture;                      // timestamps stored in microseconds - convert milli                     long absoluteframetimestampmilliseconds = picture.gettimestamp() / 1000;                     long relativeframetimestampmilliseconds = (absoluteframetimestampmilliseconds - firsttimestampmilliseconds);                     long frametimedelta = relativeframetimestampmilliseconds - playtimemilliseconds;                      if(frametimedelta > 0) {                         // video ahead of playhead, don't read more frames until catches                         sleeptimeoutmilliseconds = frametimedelta + sleeptollerancemilliseconds;                         return false;                     }                      /* if resampler not null, means didn't video in                      * bgr24 format , need convert bgr24 format                      */                     if (resampler != null) {                         // resample frame                         newpic = ivideopicture.make(                             resampler.getoutputpixelformat(),                             picture.getwidth(), picture.getheight()                         );                          if (resampler.resample(newpic, picture) < 0) {                             throw new runtimeexception("could not resample video");                         }                     }                      if (newpic.getpixeltype() != ipixelformat.type.bgr24) {                         throw new runtimeexception("could not decode video" + " bgr 24 bit data");                     }                      // , finally, convert bgr24 java buffered image                     bufferedimage javaimage = utils.videopicturetoimage(newpic);                      // update current texture                     updatetexture(javaimage);                      // let caller know texture has changed                     return true;                 }             }         }         else if(packet.getstreamindex() == this.audiostreamid)         {             iaudiosamples samples = iaudiosamples.make(1024, audiocoder.getchannels());             thread thread = new thread(new decodesoundrunnable(samples));             thread.setpriority(thread.max_priority);             this.decodethreadpool.execute(thread);          }      } 

here's audio thread:

private class decodesoundrunnable implements runnable     {         iaudiosamples samples;         int offset = 0;         istreamcoder coder;          public decodesoundrunnable(iaudiosamples samples)         {             this.samples = samples.copyreference();             this.coder = audiocoder.copyreference();         }          @override         public void run() {             while(offset < packet.getsize())             {                  int bytesdecoded = this.coder.decodeaudio(samples, packet, offset);                  if (bytesdecoded < 0)                     break;//throw new runtimeexception("got error decoding audio in: " + videopath);                   offset += bytesdecoded;                               }             playjavasound(samples, 0);             //writeoutthreadpool.execute(new writeoutsoundrunnable(samples, 0));          }     } 

solved making dedicated thread only writes out audio data. works because mline.write(byte[] bytes) block while it's writing data.

private class writeoutsoundbytes implements runnable     {         byte[] rawbyte;         public writeoutsoundbytes(byte[] rawbytes)         {             rawbyte = rawbytes;         }         @override         public void run()          {             mline.write(rawbyte, 0, rawbyte.length);         }       }