I'm using Android SDK to publish and play flashphoner streams. Some of those streams are audio only. I can successfully publish such stream, and, maybe (didn't try) I can play it with usual SurfaceViewRenderer. However, I'd like to play it without SurfaceViewRenderer (since there is no video anyway), is it possible? Also, is it possible to somehow access raw audio data? I'd like to display something like wave visualization, and for that I need access to the raw audio data while it's being received (and played) from flashphoner server (via WebRTC I guess).
Android incoming\outgoing stream audio data access
- Thread starter Evgenii
- Start date
Good day.
1) publishing audio only stream using MediaDevices example in desktop browser
2) playing audio only stream using Streaming-min Android SDK example
Yes, you can play audio only stream. This can be tested by:
1) publishing audio only stream using MediaDevices example in desktop browser
2) playing audio only stream using Streaming-min Android SDK example
No, you need SurfaceViewRenderer to play any stream. You can play with layout to hide the renderer behind a thumbnail for example.
On client side, you cannot access raw audio data. But you can get sound amplitude (see this example) and implement visualization on that basis. Please check some of sources available to capture sound.
Thanks, but SoundMeter class from your example seems to be using fresh instance of MediaRecorder, which writes to a file and not publishes that recorded sound to flashphoner stream (as far as I understand). Also, I also need to visualize sound coming in (via flashphoner stream.play()) and not only sound being published. I tried to use built-in android Visualizer class, but it doesn't work either. If I pass audio session id 0 (that is - mixed output, what you actually hear from the phone) - it doesn't provide any useful data, probably because WebRTC audio track has usage of VOICE_COMMUNICATION, denying recording. Maybe if I could get audio session id of WebRTC audio track (if there is one) - I could use Visualizer with this specific session id (instead of 0), but I didn't manage to find a way to get this session id either. I also didn't find a way to change WebRTC audio track usage to something other that VOICE_COMMUNICATION (to at least test my assumption).
If it will turn out to be not possible on client - can I get real-time access to published stream audio data on server? So that I can analyze it and feed data back to client?
If it will turn out to be not possible on client - can I get real-time access to published stream audio data on server? So that I can analyze it and feed data back to client?
We're working on raw PCM audio extraction on server side in ticket WCS-2863. But you have to use some backend communication to pass data parsed to client.