AudioStream [PCM audio in Unity]
Martin Cvengroš AudioStream provides proper PCM audio streaming - both inputs and outputs oriented - support for Unity.
It can help you with streaming from any connected system device (microphones, Line-In inputs, loopback interfaces,..) and decoding audio files and streams from local file system and network.
Please be aware that somehow advanced knowledge of Unity audio is required and is not aimed at novice users in general.
You can download demo builds from the following links to test all functionality first - they contain testing scenes for all included components; most of the scenes allow custom parameters such as filepath/urls to be set in order to test with your audio assets:
Demo builds based on v 2.4.4, Unity 2020.2.1 -
Windows x64
macOS Universal
Android
Support forum is here, but please read below first (feel free to contact me also via email or twitter)
AudioStream can stream and play from:
* internet radios M3U/8 and PLS playlists, with support for tags (+)
* audio files hosted on the internet/cloud such as podcasts
* local audio files on a regular filesystem (according to platform)
* non realtime and realtime streams from network / filesystem with caching for later usage, realtime streams can optionally play back while downloading
* memory locations containing encoded audio
* any connected input audio device (Microphone, Line-In inputs) including multichannel ones
* non default devices on iOS, such as AirPods
* any input/microphone directly into spatialized Resonance AudioSource
Additionally it can
- stream/push any of the above into connected Icecast mountpoint
- stream between two LAN instances (currently) Opus encoded audio up to two channels
- play/redirect common Unity AudioSource to other than default system output, targeting specific channel/s/speakers
- play FMOD sounds directly on specified output and its channels/speakers using customized mix matrix
[FMOD sounds means audio files in all formats supported by FMOD, including multichannel ones which Unity currently can't import - either from local disk or from network URI, using FMOD networking only (i.e. no HTTPS)]
- redirect Unity MixerGroup to other than default system output with reasonable latency (++)
- has network proxy support via UnityWebRequest
For streaming and access to audio devices only the low/Core level API of FMOD Studio is used directly where appropriate (please note that no authoring FMOD Studio functionality is used).
- for list of supported audio formats please see formats and platforms FMOD recognizes (see e.g. Wikipedia or FMOD Documentation, for example on iOS and Android m4a/AAC is supported).
It aims at zero GC allocation at runtime and is suitable for long running automatically recoverable streaming.
For network retrieval is used Unity provided UnityWebRequest - that means all sensible connection scenarios such as secure connections, redirects, custom headers and proxies,... supported by it are available.
Note: being streaming oriented it does not provide typical offline audio processing such as advanced audio clip offline signal manipulation - these are reserved for Unity AudioClip where appropriate.
AudioStream playback component can play from/a precise position and can seek within the file if DISK cache type is used, and the file being played has finite (reasonable) size.
Unity and other assets compatibility:
- compatible with all other audio packages when using AudioSource enabled, and networked components
- corresponding AudioSource can be used as usual including Unity spatialization, effecting and mixer routing - this means it can be used as AudioSource for e.g. Resonance/Oculus/Steam spatialization plugins, or any Unity Mixer in general (+++)
- fully IL2CPP compatible including standalones (Unity 2018.1 and up)
- sources are still compatible with .NET 3.5 profile
- has no PM dependencies on other optional Unity packages, not even builtin UGUI or TextMeshPro
List of included components:
AudioStream - can stream supported formats from networked or local filesystem sources and play it via Unity AudioSource
AudioStreamMinimal - same as above, but plays the audio saparately via FMOD, not going through Unity audio system.
AudioStreamDownload - can download at faster than realtime playback speeds, caches decoded PCM data which can be retrieved and played back later as an AudioClip without touching the network, optionally can be playing audio while downloading if set to be downloading at realtime speed
AudioStreamMemory - decodes audio data stored at IntPtr memory location and plays them as an AudioClip, can use cache with user identifier to retrieve immediately later
IcecastSource - pushes playing Unity AudioSource content encoded as uncompressed PCM, OGGOPUS, or OGGVORBIS into opened Icecast mountpoint
AudioSourceOutputDevice - redirects playing Unity AudioSource to specified/non default system output
- can perform multichannel separation of output, demo scene shows how to play a MONO AudioClip on selected channel of multichannel output device
AudioStreamOutputDevice native mixer plugin - similarly redirects AudioMixerGroup to specified/non default system output
AudioStreamInput - streams any connected input device into an AudioSource including multichannel microphones
- can open and stream from [loopback] devices meaning you can e.g. capture currently being played overall audio on system output on standalone platforms (which have loopback or are setup to be available)
AudioStreamInput2D - same as above, except is optimized for latency and doesn't go through Unity spatialization
AudioStreamInput_iOS - can stream from all connected devices on iOS into an Unity AudioSource, including e.g. AirPods
ResonanceSource - positional streamed audio using FMOD's provided Google Resonance native plugin
ResonanceSoundfield - can play a- and b- format ambisonics files using FMOD's provided Google Resonance native plugin
ResonanceInput - provides spatialized microphone input - streams any audio input similar to AudioStreamInput components to 3D positioned AudioSource using FMOD's provided Google Resonance native plugin
AudioStreamNetMQSource - NetMQ transport for AudioStreamNetworkSource - acts as network source for client component providing Opus encoded (max. 2-channel) stream over NetMQ transport
AudioStreamNetMQClient - NetMQ transport for AudioStreamNetworkClient - LAN client for the above
GOAudioSaveToFile - saves audio of any AudioSource into a PCM16 WAV file
AudioSourceCaptureBuffer - small utility component for exposing audio filter buffer of a game object
MediaSourceOutputDevice - direct FMOD API playback not using Unity audio system - an audio file can be played on any available system output, and can play multichannel audio files - two demo scenes are included for this component
Please read included !!README1ST!!.txt after importing the package for the first time
Included Documentation contains comprehensive install and usage guides such as building and recording on iOS, basic support for basic iOS/Android background audio modes (++++), considerations regarding Unity latency modes and DSP buffer settings, setting up Icecast source and mountpoint and each relevant setting is fully documented in the Editor via e.g. member fields tooltips.
Runs almost on all platforms Unity currently has audio support for (+++++), at the time being tested on Windows, macOS and mobiles - iOS and Android.
There is a separate test/demo scene for each component/functionality and
all source code is included.
Source is Unity API upgrader friendly (i.e. should not trigger any automatic source upgrades) and has no warnings once everything is set up.
* The whole package is as much unobtrusive as possible, there are no Editor customizations, except custom inspectors.
–––––––––––––––––––––––––––
(+) Please note that M3U/8 support is currently limited to nonrecursive, nonchunked, direct streams only (in other words HLS is not supported)
(++) Unity AudioMixer native plugin is currently built for 64- and 32-bit Windows and macOS, sources are included
(+++) Unity Mixer works with all components, except AudioSourceOutpuDevice (redirection to non default device and multichannel output separation)
(++++) User modifications of generated Xcode/Android Studio projects are required based on freely available information on the internet / e.g. RemoteCommandCenter is not included and must be implemented by user on iOS to get full player experience /
(+++++) FMOD Studio Unity Integration officially has support for Android, macOS, iOS, tvOS, Metro, UWP, Windows and Linux (both x86 and x86_64) in publicly available version, with access to consoles version available upon request.
Please note that FMOD functionality required by this asset is not currently supported on WebGL due to - for now - missing threading support.
–––––––––––––––––––––––––––
Uses FMOD Studio by Firelight Technologies
NOTE: Agreement to FMOD EULA is required before downloading the FMOD Unity Intergrations package directly from FMOD (they have very friendly indie policy though).
FMOD Unity Integration package available on this Asset Store is supported, though might be occasionally not up to date.
Please note that due to recent changes at least FMOD 2.04.01 and up is required
–––––––––––––––––––––––––––
Submitted with 2018 LTS Unity version
sources are not checked for compatibility with previous versions extensively, but initially 5.3.5 was recommended minimal version for mobiles, and earlier versions starting with 5.0 were OK for standalones.
Icecast logo By Xiph.Org - wiki.xiph.org/Logos, CC BY 3.0, https://commons.wikimedia.org/w/index.php?curid=52488057
It can help you with streaming from any connected system device (microphones, Line-In inputs, loopback interfaces,..) and decoding audio files and streams from local file system and network.
Please be aware that somehow advanced knowledge of Unity audio is required and is not aimed at novice users in general.
You can download demo builds from the following links to test all functionality first - they contain testing scenes for all included components; most of the scenes allow custom parameters such as filepath/urls to be set in order to test with your audio assets:
Demo builds based on v 2.4.4, Unity 2020.2.1 -
Windows x64
macOS Universal
Android
Support forum is here, but please read below first (feel free to contact me also via email or twitter)
AudioStream can stream and play from:
* internet radios M3U/8 and PLS playlists, with support for tags (+)
* audio files hosted on the internet/cloud such as podcasts
* local audio files on a regular filesystem (according to platform)
* non realtime and realtime streams from network / filesystem with caching for later usage, realtime streams can optionally play back while downloading
* memory locations containing encoded audio
* any connected input audio device (Microphone, Line-In inputs) including multichannel ones
* non default devices on iOS, such as AirPods
* any input/microphone directly into spatialized Resonance AudioSource
Additionally it can
- stream/push any of the above into connected Icecast mountpoint
- stream between two LAN instances (currently) Opus encoded audio up to two channels
- play/redirect common Unity AudioSource to other than default system output, targeting specific channel/s/speakers
- play FMOD sounds directly on specified output and its channels/speakers using customized mix matrix
[FMOD sounds means audio files in all formats supported by FMOD, including multichannel ones which Unity currently can't import - either from local disk or from network URI, using FMOD networking only (i.e. no HTTPS)]
- redirect Unity MixerGroup to other than default system output with reasonable latency (++)
- has network proxy support via UnityWebRequest
For streaming and access to audio devices only the low/Core level API of FMOD Studio is used directly where appropriate (please note that no authoring FMOD Studio functionality is used).
- for list of supported audio formats please see formats and platforms FMOD recognizes (see e.g. Wikipedia or FMOD Documentation, for example on iOS and Android m4a/AAC is supported).
It aims at zero GC allocation at runtime and is suitable for long running automatically recoverable streaming.
For network retrieval is used Unity provided UnityWebRequest - that means all sensible connection scenarios such as secure connections, redirects, custom headers and proxies,... supported by it are available.
Note: being streaming oriented it does not provide typical offline audio processing such as advanced audio clip offline signal manipulation - these are reserved for Unity AudioClip where appropriate.
AudioStream playback component can play from/a precise position and can seek within the file if DISK cache type is used, and the file being played has finite (reasonable) size.
Unity and other assets compatibility:
- compatible with all other audio packages when using AudioSource enabled, and networked components
- corresponding AudioSource can be used as usual including Unity spatialization, effecting and mixer routing - this means it can be used as AudioSource for e.g. Resonance/Oculus/Steam spatialization plugins, or any Unity Mixer in general (+++)
- fully IL2CPP compatible including standalones (Unity 2018.1 and up)
- sources are still compatible with .NET 3.5 profile
- has no PM dependencies on other optional Unity packages, not even builtin UGUI or TextMeshPro
List of included components:
AudioStream - can stream supported formats from networked or local filesystem sources and play it via Unity AudioSource
AudioStreamMinimal - same as above, but plays the audio saparately via FMOD, not going through Unity audio system.
AudioStreamDownload - can download at faster than realtime playback speeds, caches decoded PCM data which can be retrieved and played back later as an AudioClip without touching the network, optionally can be playing audio while downloading if set to be downloading at realtime speed
AudioStreamMemory - decodes audio data stored at IntPtr memory location and plays them as an AudioClip, can use cache with user identifier to retrieve immediately later
IcecastSource - pushes playing Unity AudioSource content encoded as uncompressed PCM, OGGOPUS, or OGGVORBIS into opened Icecast mountpoint
AudioSourceOutputDevice - redirects playing Unity AudioSource to specified/non default system output
- can perform multichannel separation of output, demo scene shows how to play a MONO AudioClip on selected channel of multichannel output device
AudioStreamOutputDevice native mixer plugin - similarly redirects AudioMixerGroup to specified/non default system output
AudioStreamInput - streams any connected input device into an AudioSource including multichannel microphones
- can open and stream from [loopback] devices meaning you can e.g. capture currently being played overall audio on system output on standalone platforms (which have loopback or are setup to be available)
AudioStreamInput2D - same as above, except is optimized for latency and doesn't go through Unity spatialization
AudioStreamInput_iOS - can stream from all connected devices on iOS into an Unity AudioSource, including e.g. AirPods
ResonanceSource - positional streamed audio using FMOD's provided Google Resonance native plugin
ResonanceSoundfield - can play a- and b- format ambisonics files using FMOD's provided Google Resonance native plugin
ResonanceInput - provides spatialized microphone input - streams any audio input similar to AudioStreamInput components to 3D positioned AudioSource using FMOD's provided Google Resonance native plugin
AudioStreamNetMQSource - NetMQ transport for AudioStreamNetworkSource - acts as network source for client component providing Opus encoded (max. 2-channel) stream over NetMQ transport
AudioStreamNetMQClient - NetMQ transport for AudioStreamNetworkClient - LAN client for the above
GOAudioSaveToFile - saves audio of any AudioSource into a PCM16 WAV file
AudioSourceCaptureBuffer - small utility component for exposing audio filter buffer of a game object
MediaSourceOutputDevice - direct FMOD API playback not using Unity audio system - an audio file can be played on any available system output, and can play multichannel audio files - two demo scenes are included for this component
Please read included !!README1ST!!.txt after importing the package for the first time
Included Documentation contains comprehensive install and usage guides such as building and recording on iOS, basic support for basic iOS/Android background audio modes (++++), considerations regarding Unity latency modes and DSP buffer settings, setting up Icecast source and mountpoint and each relevant setting is fully documented in the Editor via e.g. member fields tooltips.
Runs almost on all platforms Unity currently has audio support for (+++++), at the time being tested on Windows, macOS and mobiles - iOS and Android.
There is a separate test/demo scene for each component/functionality and
all source code is included.
Source is Unity API upgrader friendly (i.e. should not trigger any automatic source upgrades) and has no warnings once everything is set up.
* The whole package is as much unobtrusive as possible, there are no Editor customizations, except custom inspectors.
–––––––––––––––––––––––––––
(+) Please note that M3U/8 support is currently limited to nonrecursive, nonchunked, direct streams only (in other words HLS is not supported)
(++) Unity AudioMixer native plugin is currently built for 64- and 32-bit Windows and macOS, sources are included
(+++) Unity Mixer works with all components, except AudioSourceOutpuDevice (redirection to non default device and multichannel output separation)
(++++) User modifications of generated Xcode/Android Studio projects are required based on freely available information on the internet / e.g. RemoteCommandCenter is not included and must be implemented by user on iOS to get full player experience /
(+++++) FMOD Studio Unity Integration officially has support for Android, macOS, iOS, tvOS, Metro, UWP, Windows and Linux (both x86 and x86_64) in publicly available version, with access to consoles version available upon request.
Please note that FMOD functionality required by this asset is not currently supported on WebGL due to - for now - missing threading support.
–––––––––––––––––––––––––––
Uses FMOD Studio by Firelight Technologies
NOTE: Agreement to FMOD EULA is required before downloading the FMOD Unity Intergrations package directly from FMOD (they have very friendly indie policy though).
FMOD Unity Integration package available on this Asset Store is supported, though might be occasionally not up to date.
Please note that due to recent changes at least FMOD 2.04.01 and up is required
–––––––––––––––––––––––––––
Submitted with 2018 LTS Unity version
sources are not checked for compatibility with previous versions extensively, but initially 5.3.5 was recommended minimal version for mobiles, and earlier versions starting with 5.0 were OK for standalones.
Icecast logo By Xiph.Org - wiki.xiph.org/Logos, CC BY 3.0, https://commons.wikimedia.org/w/index.php?curid=52488057
