Diffuser en direct Android Audio au serveur

Question

J'essaie actuellement de diffuser l'audio en direct d'un microphone d'un Android périphérique vers un programme Java. J'ai commencé par envoyer le son en direct entre deux Android appareils pour confirmer ma méthode était correcte. L'audio pouvait être parfaitement entendu avec à peine tout retard sur l'appareil récepteur. Ensuite, j'envoie le même flux audio à un petit Java programme et j’ai vérifié que les données étaient correctement envoyées ici. Maintenant, ce que je veux faire est de coder ces données et de les lire en quelque sorte sur le serveur qui exécute le programme Java. Je préférerais jouer dans un navigateur Web utilisant HTML5 ou JavaScript, mais je suis ouvert aux méthodes alternatives telles que VLC.

Voici le code de l'application Android qui envoie le son du microphone en direct

public class MainActivity extends Activity {


private Button startButton,stopButton;

public byte[] buffer;
public static DatagramSocket socket;
    AudioRecord recorder;

private int sampleRate = 44100;   
private int channelConfig = AudioFormat.CHANNEL_CONFIGURATION_MONO;    
private int audioFormat = AudioFormat.ENCODING_PCM_16BIT;       
int minBufSize = AudioRecord.getMinBufferSize(sampleRate, channelConfig, audioFormat);
    private boolean status = true;

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);

     startButton = (Button) findViewById (R.id.start_button);
     stopButton = (Button) findViewById (R.id.stop_button);

     startButton.setOnClickListener(startListener);
     stopButton.setOnClickListener(stopListener);

     minBufSize += 2048;
}

@Override
public boolean onCreateOptionsMenu(Menu menu) {
    getMenuInflater().inflate(R.menu.main, menu);
    return true;
}

private final OnClickListener stopListener = new OnClickListener() {

    @Override
    public void onClick(View arg0) {
                status = false;
                recorder.release();
                Log.d("VS","Recorder released");
    }
};

private final OnClickListener startListener = new OnClickListener() {

    @Override
    public void onClick(View arg0) {
                status = true;
                startStreaming();           
    }
};



public void startStreaming()
{
    Thread streamThread = new Thread(new Runnable(){
        @Override
        public void run()
        {
            try{

                DatagramSocket socket = new DatagramSocket();
                Log.d("VS", "Socket Created");

                byte[] buffer = new byte[minBufSize];

                Log.d("VS","Buffer created of size " + minBufSize);


                Log.d("VS", "Address retrieved");
                recorder = new AudioRecord(MediaRecorder.AudioSource.MIC,sampleRate,channelConfig,audioFormat,minBufSize);
                Log.d("VS", "Recorder initialized");


                recorder.startRecording();


                InetAddress IPAddress = InetAddress.getByName("192.168.1.5");
                byte[] sendData = new byte[1024];
                byte[] receiveData = new byte[1024];


                while (status == true)
                {
                    DatagramPacket sendPacket = new DatagramPacket(sendData, sendData.length, IPAddress, 50005);
                    socket.send(sendPacket);
                }

            } catch(UnknownHostException e) {
                Log.e("VS", "UnknownHostException");
            } catch (IOException e) {
                Log.e("VS", "IOException");
                e.printStackTrace();
            } 


        }

    });
    streamThread.start();
}
}

Et voici le code pour la lecture du programme Java dans les données ..

class Server
{
   public static void main(String args[]) throws Exception
      {
         DatagramSocket serverSocket = new DatagramSocket(50005);
            byte[] receiveData = new byte[1024];
            byte[] sendData = new byte[1024];
            while(true)
               {
                  DatagramPacket receivePacket = new DatagramPacket(receiveData, receiveData.length);



              serverSocket.receive(receivePacket);
              String sentence = new String( receivePacket.getData().toString());

              System.out.println("RECEIVED: " + sentence);
           }
  }
}

Je sais que je devrais encoder l'audio du côté de l'application avant de l'envoyer au programme Java), mais je ne suis pas sûr de savoir comment procéder à l'encodage lors de l'utilisation de AudioRecorder. Je préférerais ne pas utiliser NDK comme je n’ai aucune expérience et n’ai pas vraiment le temps d’apprendre à l’utiliser .... pour le moment :)

Answer 1

Alors j'ai résolu mon problème. Le problème était principalement du côté de la réception. Le récepteur capte le flux audio et le transmet aux haut-parleurs du PC. La voix qui en résulte est toujours assez lente et cassée, mais cela fonctionne néanmoins. Jouer avec la taille du tampon peut améliorer cela.

Edit: vous utilisez un fil pour lire l'audio afin d'éviter le décalage. En outre, il est préférable d’utiliser une taille d’échantillonnage de 16 000, ce qui convient à la voix.

Code Android:

package com.example.mictest2;

import Java.io.IOException;
import Java.net.DatagramPacket;
import Java.net.DatagramSocket;
import Java.net.InetAddress;
import Java.net.UnknownHostException;

import Android.app.Activity;
import Android.media.AudioFormat;
import Android.media.AudioRecord;
import Android.media.MediaRecorder;
import Android.os.Bundle;
import Android.util.Log;
import Android.view.View;
import Android.view.View.OnClickListener;
import Android.widget.Button;

public class Send extends Activity {
private Button startButton,stopButton;

public byte[] buffer;
public static DatagramSocket socket;
private int port=50005;

AudioRecord recorder;

private int sampleRate = 16000 ; // 44100 for music
private int channelConfig = AudioFormat.CHANNEL_CONFIGURATION_MONO;    
private int audioFormat = AudioFormat.ENCODING_PCM_16BIT;       
int minBufSize = AudioRecord.getMinBufferSize(sampleRate, channelConfig, audioFormat);
private boolean status = true;


@Override
public void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);

    startButton = (Button) findViewById (R.id.start_button);
    stopButton = (Button) findViewById (R.id.stop_button);

    startButton.setOnClickListener (startListener);
    stopButton.setOnClickListener (stopListener);

}

private final OnClickListener stopListener = new OnClickListener() {

    @Override
    public void onClick(View arg0) {
                status = false;
                recorder.release();
                Log.d("VS","Recorder released");
    }

};

private final OnClickListener startListener = new OnClickListener() {

    @Override
    public void onClick(View arg0) {
                status = true;
                startStreaming();           
    }

};

public void startStreaming() {


    Thread streamThread = new Thread(new Runnable() {

        @Override
        public void run() {
            try {

                DatagramSocket socket = new DatagramSocket();
                Log.d("VS", "Socket Created");

                byte[] buffer = new byte[minBufSize];

                Log.d("VS","Buffer created of size " + minBufSize);
                DatagramPacket packet;

                final InetAddress destination = InetAddress.getByName("192.168.1.5");
                Log.d("VS", "Address retrieved");


                recorder = new AudioRecord(MediaRecorder.AudioSource.MIC,sampleRate,channelConfig,audioFormat,minBufSize*10);
                Log.d("VS", "Recorder initialized");

                recorder.startRecording();


                while(status == true) {


                    //reading data from MIC into buffer
                    minBufSize = recorder.read(buffer, 0, buffer.length);

                    //putting buffer in the packet
                    packet = new DatagramPacket (buffer,buffer.length,destination,port);

                    socket.send(packet);
                    System.out.println("MinBufferSize: " +minBufSize);


                }



            } catch(UnknownHostException e) {
                Log.e("VS", "UnknownHostException");
            } catch (IOException e) {
                e.printStackTrace();
                Log.e("VS", "IOException");
            } 
        }

    });
    streamThread.start();
 }
 }

Android XML:

<RelativeLayout xmlns:Android="http://schemas.Android.com/apk/res/Android"
xmlns:tools="http://schemas.Android.com/tools"
Android:layout_width="match_parent"
Android:layout_height="match_parent"
Android:paddingBottom="@dimen/activity_vertical_margin"
Android:paddingLeft="@dimen/activity_horizontal_margin"
Android:paddingRight="@dimen/activity_horizontal_margin"
Android:paddingTop="@dimen/activity_vertical_margin"
tools:context=".MainActivity" >

<TextView
    Android:id="@+id/textView1"
    Android:layout_width="wrap_content"
    Android:layout_height="wrap_content"
    Android:text="@string/hello_world" />

<Button
    Android:id="@+id/start_button"
    Android:layout_width="wrap_content"
    Android:layout_height="wrap_content"
    Android:layout_below="@+id/textView1"
    Android:layout_centerHorizontal="true"
    Android:layout_marginTop="130dp"
    Android:text="Start" />

<Button
    Android:id="@+id/stop_button"
    Android:layout_width="wrap_content"
    Android:layout_height="wrap_content"
    Android:layout_alignLeft="@+id/button1"
    Android:layout_below="@+id/button1"
    Android:layout_marginTop="64dp"
    Android:text="Stop" />

</RelativeLayout>

Code serveur:

package com.datagram;

import Java.io.ByteArrayInputStream;
import Java.net.DatagramPacket;
import Java.net.DatagramSocket;

import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioInputStream;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.DataLine;
import javax.sound.sampled.FloatControl;
import javax.sound.sampled.SourceDataLine;

class Server {

AudioInputStream audioInputStream;
static AudioInputStream ais;
static AudioFormat format;
static boolean status = true;
static int port = 50005;
static int sampleRate = 44100;

public static void main(String args[]) throws Exception {


    DatagramSocket serverSocket = new DatagramSocket(50005);


    byte[] receiveData = new byte[1280]; 
    // ( 1280 for 16 000Hz and 3584 for 44 100Hz (use AudioRecord.getMinBufferSize(sampleRate, channelConfig, audioFormat) to get the correct size)

    format = new AudioFormat(sampleRate, 16, 1, true, false);

    while (status == true) {
        DatagramPacket receivePacket = new DatagramPacket(receiveData,
                receiveData.length);

        serverSocket.receive(receivePacket);

        ByteArrayInputStream baiss = new ByteArrayInputStream(
                receivePacket.getData());

        ais = new AudioInputStream(baiss, format, receivePacket.getLength());

        // A thread solve the problem of chunky audio 
        new Thread(new Runnable() {
            @Override
            public void run() {
                toSpeaker(receivePacket.getData(), sourceDataLine);
            }
        }).start();
    }
}

public static void toSpeaker(byte soundbytes[]) {
    try {

        DataLine.Info dataLineInfo = new DataLine.Info(SourceDataLine.class, format);
        SourceDataLine sourceDataLine = (SourceDataLine) AudioSystem.getLine(dataLineInfo);

        sourceDataLine.open(format);

        FloatControl volumeControl = (FloatControl) sourceDataLine.getControl(FloatControl.Type.MASTER_GAIN);
        volumeControl.setValue(100.0f);

        sourceDataLine.start();
        sourceDataLine.open(format);

        sourceDataLine.start();

        System.out.println("format? :" + sourceDataLine.getFormat());

        sourceDataLine.write(soundbytes, 0, soundbytes.length);
        System.out.println(soundbytes.toString());
        sourceDataLine.drain();
        sourceDataLine.close();
    } catch (Exception e) {
        System.out.println("Not working in speakers...");
        e.printStackTrace();
    }
}
}

J'espère que cela vous aidera à économiser quelques heures de douleur :)

Answer 2

Mes 2 centimes à votre code pour améliorer l'efficacité. Bien essayé

package com.datagram;

import Java.io.ByteArrayInputStream;
import Java.net.DatagramPacket;
import Java.net.DatagramSocket;

import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioInputStream;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.DataLine;
import javax.sound.sampled.FloatControl;
import javax.sound.sampled.SourceDataLine;

class Server {

AudioInputStream audioInputStream;
static AudioInputStream ais;
static AudioFormat format;
static boolean status = true;
static int port = 50005;
static int sampleRate = 44100;

static DataLine.Info dataLineInfo;
static SourceDataLine sourceDataLine;

public static void main(String args[]) throws Exception {

    DatagramSocket serverSocket = new DatagramSocket(port);

    /**
     * Formula for lag = (byte_size/sample_rate)*2
     * Byte size 9728 will produce ~ 0.45 seconds of lag. Voice slightly broken.
     * Byte size 1400 will produce ~ 0.06 seconds of lag. Voice extremely broken.
     * Byte size 4000 will produce ~ 0.18 seconds of lag. Voice slightly more broken then 9728.
     */

    byte[] receiveData = new byte[4096];

    format = new AudioFormat(sampleRate, 16, 1, true, false);
    dataLineInfo = new DataLine.Info(SourceDataLine.class, format);
    sourceDataLine = (SourceDataLine) AudioSystem.getLine(dataLineInfo);
    sourceDataLine.open(format);
    sourceDataLine.start();

    FloatControl volumeControl = (FloatControl) sourceDataLine.getControl(FloatControl.Type.MASTER_GAIN);
    volumeControl.setValue(1.00f);

    DatagramPacket receivePacket = new DatagramPacket(receiveData,
            receiveData.length);
    ByteArrayInputStream baiss = new ByteArrayInputStream(
            receivePacket.getData());
    while (status == true) {
        serverSocket.receive(receivePacket);
        ais = new AudioInputStream(baiss, format, receivePacket.getLength());
        toSpeaker(receivePacket.getData());
    }
    sourceDataLine.drain();
    sourceDataLine.close();
}

    public static void toSpeaker(byte soundbytes[]) {
        try {
            sourceDataLine.write(soundbytes, 0, soundbytes.length);
        } catch (Exception e) {
            System.out.println("Not working in speakers...");
            e.printStackTrace();
        }
    }
}

Answer 3

La voix est cassée à cause de la ligne suivante dans votre Android code:

minBufSize += 2048;

Vous ajoutez simplement des octets vides. Utiliser aussi CHANNEL_IN_MONO au lieu de CHANNEL_CONFIGURATION_MONO