IotHub
  1. audio
IotHub
  • README
  • Update Record
  • token
    • createToken
      POST
    • destroyToken
      POST
  • customer
    • customerPage
      POST
    • customerTree
      POST
  • device
    • command
      • commandSend
      • commandInfoList
      • cutOilElectric
      • getConfig
    • devicePage
      POST
    • deviceTypes
      POST
    • deviceTypeInfo
      POST
    • deviceDataByDeviceIds
      POST
    • deviceDataByCustomerId
      POST
    • deviceTripInfoByTripId
      POST
    • deviceTripsByDeviceId
      POST
    • expandInfoByDeviceId
      POST
    • saveDeviceChannelName
      POST
  • fence
    • fencePage
      POST
    • fenceInfo
      POST
    • addFence
      POST
    • saveFence
      POST
    • delFence
      POST
    • fenceDevices
      POST
    • fenceDeviceUpdate
      POST
    • fenceDeviceDel
      POST
    • fenceDeviceAdd
      POST
  • push
    • README
  • video
    • H5 page
    • H5 Page-V2
    • liveVideo
    • playBackList
    • historicalUpload
    • playBack
    • replaySet
    • taskPage
    • taskDelete
    • getNodes
  • alarm
    • alarmTypeList
    • alarmFilePage
    • alarmFileByAlarmId
    • alarmHistoryPage
    • alarmPageByUser
  • audio
    • README
    • talk
      POST
    • talkPlus
      POST
  • gallery
    • filePageV2
    • filePage
    • deleteFile
    • capture
    • capturePage
  • fleet
    • faceRecordPage
  • gps
    • historyData
  • Protocol Analysis
    POST
  1. audio

README

Flow#

image.png

Demo#

https://github.com/mettaxIot/twoWayTalk

1、Call the HTTP interface to obtain the Token#

/v2/openapi/audio/talk
/v2/openapi/audio/talk/plus

2.create WebSocket#

talkNode#

Different nodes will have different call addresses
标题
Defaultwss://m1.mettaxiot.com/talk/websocket/
indiawss://india.mettaxiot.com/talk/websocket/
uaewss://uae.mettaxiot.com/talk/websocket/
southafricawss://southafrica.mettaxiot.com/talk/websocket/

param#

imei:deviceId
token:from http interface
callback:

back#

image.png
msg.data instanceof Blob: Audio data
let data = JSON.parse(msg.data);
data.code === 'connect':Link successfully established
data.code === 'repeat':Repeated connections indicate that a new user has initiated a conversation with the device, and the current link has expired
data.code === 'release':The device has disconnected from the link
data.code === 'network':data.deviceTime is the timestamp of data on the device (related to the time zone set by the device)

demo#

In the above example, the 'getSocket' function is used to establish a WebSocket connection with the specified device. It first constructs the URL for the WebSocket connection based on the incoming 'imei' and 'token', and then creates the 'WebSocket' object. Next, it sets up event handlers for 'onopen', 'onmessage', 'onclose', and 'oneerror' to handle situations where a connection is opened, a message is received, a connection is closed, and an error occurs, respectively.
When the connection is opened, set the 'loading' status to 'true' and print a message to the console. When receiving a message, perform corresponding operations based on the content of the message, such as receiving Blob data, displaying warning messages, or calling callback functions. When the connection is closed or an error occurs, set the 'loading' status to 'false' and print the corresponding message to the console.
Note: The '_this' variable in the example is used to access the external context (i.e.' this') within the callback function within the closure. This is because the 'this' pointer in the callback function may change, so it is usually necessary to save the reference to the external context in a variable for use in the callback function

3.receive audio#

Explain#

The following is an implementation of the 'receive' method, which is responsible for processing audio data received from WebSocket.
Audio Format:G711A

Eplain#

-This method depends on the presence of an audio element with the ID 'LogAudioPlayer' on the page.
-Assuming the 'Recorder' object is an object containing audio processing methods, and 'g711a2wav' is one of the methods used to convert G.711 encoded audio data into WAV format. In practical applications, it is necessary to ensure that the 'Recorder' object and its methods have been correctly implemented and introduced.
-The error handling in the method is relatively simple, only outputting prompt information through the console. In practical applications, more detailed error handling and user feedback may be required.

4.Microphone#

Explain#

Control the status of the microphone. When the microphone is turned on, record the current platform microphone sound and send it to the device.
Suggested parameters:
channelCount :1
bitRate:16
sampleRate:16000

demo#

This function is used to control the state of the microphone. When the 'changeMicrophone' method is called, the value of 'this. isMicrophone' is reversed, meaning that if the microphone was originally in an open state, it becomes closed, and vice versa. Subsequently, based on the current state of 'this. isMicrophone', perform the corresponding operation: if the microphone state is on, call the 'initRecord' method to initialize the recording; If the microphone status is off, call the 'closeRecord' method to turn off recording (please note that the implementation of the 'closeRecord' method is not provided in the given code snippet).
The initRecord method is used to initialize the recording function. It first creates an instance of 'Recorder' and configures the relevant parameters for recording. Among them, 'onProcess' is a callback function that is called during the recording process to perform real-time processing on the recorded audio data. In this example, audio data is passed to _this The RealTimeSendTry method may be used to send audio data in real-time to a device.
Next, call the 'this. record. open' method to open the recording device. This method accepts two callback functions as parameters: the first callback function is called after the recording device is successfully opened, and it will start recording and call _this The RealTimeSendTryReset method is used to reset the environment (possibly in preparation for sending new audio data); The second callback function handles the situation where the recording device fails to open. It will print the error message to the console. If it is because the user does not allow recording, a 'UserNotAllow' prompt will be added before the error message.
By using the methods of 'changeMicrophone' and 'initRecord', it is possible to control the microphone and initialize the recording. In practical applications, more details need to be handled, such as error handling, saving and playing audio files, etc. At the same time, it is also necessary to ensure that users have authorized recording permissions and respect their privacy.

Explain#

In the above code, the 'changeMicrophone' method is used to switch the state of the microphone. When the microphone is turned on, call the 'initRecord' method to initialize the recording function; When the microphone is turned off, call the 'closeRecord' method to stop recording and clean up related resources.
The 'initRecord' method initializes the recording function and configures recording parameters by creating an instance of the 'Recorder'. During the recording process, pause by using 'onProcess'
Modified at 2025-02-10 07:40:51
Previous
alarmPageByUser
Next
talk
Built with