Flow

1、Call the HTTP interface to obtain the Token

• /v2/openapi/audio/talk
  ``` html
  Unable to select the corresponding audio node. Not recommended, please use "/v2/openapp/audio/talk/plus“

request:
{"deviceId":"105045133913","channelId":1}

response：
{
"code": 0,
"data": "3cdd1248b12126c3594b",
"msg": ""
}

- /v2/openapi/audio/talk/plus

``` html
request:
{"deviceId":"105045133913","channelId":1,"talkChannel":"india-1"}

response：
{
    "code": 0,
    "data": {
        "deviceId": "105045133913",
        "channelId": 1,
        "talkChannel": "india-1",
        "talkUrl": "wss://india.mettaxiot.com/talk/websocket/105045133913/3cdd1248b12126c3594b"
    },
    "msg": ""
}

2.create WebSocket

talkNode

Different nodes will have different call addresses

标题
Default	wss://m1.mettaxiot.com/talk/websocket/
india	wss://india.mettaxiot.com/talk/websocket/
uae	wss://uae.mettaxiot.com/talk/websocket/
southafrica	wss://southafrica.mettaxiot.com/talk/websocket/

param

• imei：deviceId
• token：from http interface
• callback：

  back

• msg.data instanceof Blob: Audio data
• let data = JSON.parse(msg.data);
  • data.code === 'connect':Link successfully established
  • data.code === 'repeat':Repeated connections indicate that a new user has initiated a conversation with the device, and the current link has expired
  • data.code === 'release':The device has disconnected from the link
  • data.code === 'network':data.deviceTime is the timestamp of data on the device (related to the time zone set by the device)

demo

getSocket(imei, token, callback) {

    const url = 'wss://www.mettaxiot.com/talk/websocket/' + imei + '/' + token;
    //    const url = wss://m1.mettaxiot.com/talk/websocket/105045133913/3cdd1248b12126c3594b

    let _this = this;

    this.socket = new WebSocket(url);

     this.socket.onopen = function () {
        _this.loading = true;
    };
   
    this.socket.onmessage = function (msg) {
        if (msg.data instanceof Blob) {
            _this.loading = false;
            _this.receive(msg);
        } else if (msg.data === 'repeat') {
            _this.loading = false;
            _this.$message.warning(_this.$t('device busy'));
            callback();
        } else if (msg.data === 'error') {
            _this.loading = false;
            _this.$message.warning(_this.$t('device error'));
            callback();
        } else if (msg.data.code === 'network'){
        //msg.data :{"deviceTime": "1725976558122",  "serverTime": "1725947758702",  "code": "network"}
        //devicetime uses the device's time zone 
        //serverTime uses 0 time zone
        }
    };

    this.socket.onclose = function (e) {
        _this.loading = false;
        console.log(e, 'onclose');
    };

    this.socket.onerror = function () {
        _this.loading = false;
    };


}

In the above example, the 'getSocket' function is used to establish a WebSocket connection with the specified device. It first constructs the URL for the WebSocket connection based on the incoming 'imei' and 'token', and then creates the 'WebSocket' object. Next, it sets up event handlers for 'onopen', 'onmessage', 'onclose', and 'oneerror' to handle situations where a connection is opened, a message is received, a connection is closed, and an error occurs, respectively.
When the connection is opened, set the 'loading' status to 'true' and print a message to the console. When receiving a message, perform corresponding operations based on the content of the message, such as receiving Blob data, displaying warning messages, or calling callback functions. When the connection is closed or an error occurs, set the 'loading' status to 'false' and print the corresponding message to the console.
Note: The '_this' variable in the example is used to access the external context (i.e.' this') within the callback function within the closure. This is because the 'this' pointer in the callback function may change, so it is usually necessary to save the reference to the external context in a variable for use in the callback function

3.receive audio

Explain

The following is an implementation of the 'receive' method, which is responsible for processing audio data received from WebSocket.

Audio Format：G711A

receive(msg) {
  
    if (typeof msg.data === 'string') {
        console.log(msg.data); 
    }
  
    if (msg.data instanceof Blob) {

        var audio = document.getElementById('LogAudioPlayer');
        audio.volume = this.volume;  
        if (!(audio.ended || audio.paused)) {
            audio.pause();
        }
        var end = function (blob) {          
            audio.src = (window.URL || window.webkitURL).createObjectURL(blob);
            audio.play();
        };
       
        var wav = Recorder['g711a2wav'];
        if (wav) {
            var wavData = msg.data;  
            wav(
                wavData,
                function (blob) {                    
                    end(blob);
                },
                function (msg) {
                    
                }
            );
        } else {           
            console.log('end blob'); 
        }
    }
}

Eplain

-This method depends on the presence of an audio element with the ID 'LogAudioPlayer' on the page.
-Assuming the 'Recorder' object is an object containing audio processing methods, and 'g711a2wav' is one of the methods used to convert G.711 encoded audio data into WAV format. In practical applications, it is necessary to ensure that the 'Recorder' object and its methods have been correctly implemented and introduced.
-The error handling in the method is relatively simple, only outputting prompt information through the console. In practical applications, more detailed error handling and user feedback may be required.

4.Microphone

Explain

Control the status of the microphone. When the microphone is turned on, record the current platform microphone sound and send it to the device.

Suggested parameters:
channelCount :1
bitRate:16
sampleRate:16000

demo

changeMicrophone() {
    this.isMicrophone = !this.isMicrophone;
    if (this.isMicrophone) {
        this.initRecord(); 
    } else {
        this.closeRecord(); 
    }
},

This function is used to control the state of the microphone. When the 'changeMicrophone' method is called, the value of 'this. isMicrophone' is reversed, meaning that if the microphone was originally in an open state, it becomes closed, and vice versa. Subsequently, based on the current state of 'this. isMicrophone', perform the corresponding operation: if the microphone state is on, call the 'initRecord' method to initialize the recording; If the microphone status is off, call the 'closeRecord' method to turn off recording (please note that the implementation of the 'closeRecord' method is not provided in the given code snippet).

initRecord() {
    let _this = this;
    this.record = Recorder({
        type: 'unknown',
        onProcess: function (buffers, powerLevel, bufferDuration, bufferSampleRate) {
            _this.RealTimeSendTry(buffers, bufferSampleRate, false);
        }
    });

    this.record.open(
        function () {
            _this.record.start(); 
            _this.RealTimeSendTryReset(); 
        },
        function (msg, isUserNotAllow) {
            console.log((isUserNotAllow ? 'UserNotAllow，' : '') + msg);
        }
    );
}

The initRecord method is used to initialize the recording function. It first creates an instance of 'Recorder' and configures the relevant parameters for recording. Among them, 'onProcess' is a callback function that is called during the recording process to perform real-time processing on the recorded audio data. In this example, audio data is passed to _this The RealTimeSendTry method may be used to send audio data in real-time to a device.

Next, call the 'this. record. open' method to open the recording device. This method accepts two callback functions as parameters: the first callback function is called after the recording device is successfully opened, and it will start recording and call _this The RealTimeSendTryReset method is used to reset the environment (possibly in preparation for sending new audio data); The second callback function handles the situation where the recording device fails to open. It will print the error message to the console. If it is because the user does not allow recording, a 'UserNotAllow' prompt will be added before the error message.

By using the methods of 'changeMicrophone' and 'initRecord', it is possible to control the microphone and initialize the recording. In practical applications, more details need to be handled, such as error handling, saving and playing audio files, etc. At the same time, it is also necessary to ensure that users have authorized recording permissions and respect their privacy.

closeRecord() {
    if (this.record === null) {
        return;
    }

    this.record.close(); 
    this.RealTimeSendTry([], 0, true); 

    let _this = this;
    this.record.stop(
        function (blob, duration) {
            var newRec = Recorder({ type: 'g711a' });
            var pcm = Recorder.SampleData(_this.record.buffers, _this.record.srcSampleRate, _this.record.srcSampleRate).data;
            newRec.mock(pcm, _this.record.srcSampleRate);

            newRec.stop(function (blob, duration) {
                var audio = document.getElementById('LogAudioPlayer');
                audio.style.display = 'inline-block';
                if (!(audio.ended || audio.paused)) {
                    audio.pause();
                }

                var end = function (blob) {
                    audio.src = (window.URL || window.webkitURL).createObjectURL(blob);
                    audio.play(); 
                };

                var wav = Recorder['g711a2wav'];
                if (wav) {
                    var wavData = blob;
                    wav(
                        wavData,
                        function (blob) {
                            end(blob);
                        },
                        function (msg) {
                            console.error(msg);
                        }
                    );
                } else {
                    end(blob);
                }
            });
        },
        function (msg) {
            console.info(msg);
        },
        true
    );

    this.record = null;
},


RealTimeSendTryReset() {
    this.realTimeSendTryChunks = null;
},

RealTimeSendTry(buffers, bufferSampleRate, isClose) {
    let _this = this;
    if (this.realTimeSendTryChunks == null) {
        this.realTimeSendTryNumber = 0;
        this.transferUploadNumberMax = 0;
        this.realTimeSendTryChunk = null;
        this.realTimeSendTryChunks = [];
    }

    if (this.testBitRate == 16 && this.SendFrameSize % 2 == 1) {
        console.log('SendFrameSize error');
        return;
    }

    var pcm = [];
    var pcmSampleRate = 0;
    if (buffers.length > 0) {       
        var chunk = Recorder.SampleData(buffers, bufferSampleRate, this.testSampleRate, this.realTimeSendTryChunk);

        for (var i = this.realTimeSendTryChunk ? this.realTimeSendTryChunk.index : 0; i < chunk.index; i++) {
            buffers[i] = null;
        }
        this.realTimeSendTryChunk = chunk; 

        pcm = chunk.data;
        pcmSampleRate = chunk.sampleRate;

    }
   
    if (pcm.length > 0) {
        this.realTimeSendTryChunks.push({ pcm: pcm, pcmSampleRate: pcmSampleRate });
    }

    var chunkSize = this.SendFrameSize / (this.testBitRate / 8); 

    pcm = new Int16Array(chunkSize);
    pcmSampleRate = 0;
    var pcmOK = false;
    var pcmLen = 0;
    for1: for (var i1 = 0; i1 < this.realTimeSendTryChunks.length; i1++) {
        chunk = this.realTimeSendTryChunks[i1];
        pcmSampleRate = chunk.pcmSampleRate;

        for (var i2 = chunk.offset || 0; i2 < chunk.pcm.length; i2++) {
            pcm[pcmLen] = chunk.pcm[i2];
            pcmLen++;

            if (pcmLen == chunkSize) {
                pcmOK = true;
                chunk.offset = i2 + 1;
                for (var i3 = 0; i3 < i1; i3++) {
                    this.realTimeSendTryChunks.splice(0, 1);
                }
                break for1;
            }
        }
    }
    
    if (!pcmOK) {
        if (isClose) {
            var number = ++this.realTimeSendTryNumber;
            this.TransferUpload(number, null, 0, null, isClose);
        }
        return;
    }

    number = ++this.realTimeSendTryNumber;
    var encStartTime = Date.now();

    var recMock = Recorder({
        type: 'g711a',
        sampleRate: this.testSampleRate, 
        bitRate: this.testBitRate 
    });
    recMock.mock(pcm, pcmSampleRate);
    recMock.stop(
        function (blob, duration) {
            blob.encTime = Date.now() - encStartTime;
            _this.TransferUpload(number, blob, duration, recMock, false);
            _this.RealTimeSendTry([], 0, isClose);
        },
        function (msg) {
            console.log(msg);
        }
    );
},

TransferUpload(number, blobOrNull, duration, blobRec, isClose) {
    this.transferUploadNumberMax = Math.max(this.transferUploadNumberMax, number);
    if (blobOrNull) {
        var blob = blobOrNull;
        // var encTime = blob.encTime;
        this.socket.send(blob);
    }
    if (isClose) {
        console.log('No.' + (number < 100 ? ('000' + number).substr(-3) : number) + ':stop');
    }
},

Explain

In the above code, the 'changeMicrophone' method is used to switch the state of the microphone. When the microphone is turned on, call the 'initRecord' method to initialize the recording function; When the microphone is turned off, call the 'closeRecord' method to stop recording and clean up related resources.

The 'initRecord' method initializes the recording function and configures recording parameters by creating an instance of the 'Recorder'. During the recording process, pause by using 'onProcess'