Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add audio engine v2 #15839

Draft
wants to merge 46 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
8c84ad8
Add audio engine v2
docEdub Nov 18, 2024
125e067
Fix `TransformNode` imports
docEdub Nov 18, 2024
570568d
Make `CreateSound...` function `engine` args optional
docEdub Nov 19, 2024
4ad0854
Rename v2/audioEngine.ts to v2/audioEngineV2.ts
docEdub Nov 20, 2024
0e6af20
Improve audio context unlock handling
docEdub Nov 20, 2024
1588a23
Don't resolve `WebAudioStreamingSound.init` until document has a body
docEdub Nov 20, 2024
32ea7f9
Remove unused `WebAudioEngine._resolveInitPromise`
docEdub Nov 20, 2024
770d61c
Fix `WebAudioStreamingSoundInstance` start offset issue
docEdub Nov 20, 2024
a26abf3
Cleanup order of `WebAudioStaticSoundInstance` members
docEdub Nov 20, 2024
69c2831
Don't return from `WebAudioEngine.init` until `isReadyPromise` resolves
docEdub Nov 21, 2024
33bcefd
Move v2/index export from src/index.ts to src/Audio/index.ts
docEdub Nov 21, 2024
8d7b35c
Add underscore to internal classes to hide them from BABYLON namespace
docEdub Nov 21, 2024
dd8ba78
Update `getClassName()` strings
docEdub Nov 21, 2024
4d05622
Expand static/streaming sound source parameter type for discoverability
docEdub Nov 21, 2024
7d21bb3
Swap order of `CreateSound...` engine and options args so engine is last
docEdub Nov 21, 2024
d5b76cd
Resolve WebAudio engine `isReadyPromise` when done initializing
docEdub Nov 21, 2024
a8e734a
Export sound source types
docEdub Nov 22, 2024
15eb6df
Make `_CreateMainAudioOutputAsync` internal
docEdub Nov 22, 2024
cb88f16
Add underscore to internal classes to hide them from BABYLON namespace
docEdub Nov 22, 2024
1aee57c
Add audio engine `unlock()`
docEdub Nov 25, 2024
1082bd4
Add preload instance API to streaming sound classes
docEdub Nov 25, 2024
80c1ee3
Remove `WebAudioEngine.isUnlockedPromise`
docEdub Nov 25, 2024
ffeb598
Don't return sound instances from sound `play` functions
docEdub Nov 26, 2024
1d0f02c
Add underscore prefix to internal sound instance classes
docEdub Nov 26, 2024
abb88f7
Put streaming sound options in alphabetical order and expand string type
docEdub Nov 26, 2024
e4fc35c
Fix static sound file extension regex
docEdub Nov 26, 2024
57da08a
Add audio engine `volume` property
docEdub Nov 26, 2024
248f298
Disable `no-internal-modules` check in Audio/index.ts
docEdub Nov 26, 2024
eaf7362
Fix static sound file extension regex for URLs with no query parameters
docEdub Nov 26, 2024
3e6ad6c
Remove unnecessary `bind()` calls
docEdub Nov 26, 2024
dc42780
Fix play/pause/resume on static and streaming sounds
docEdub Nov 26, 2024
19b01a6
Fix streaming sound state issues when audio context is not running
docEdub Dec 4, 2024
e41c85d
Don't add streaming sound media element to document body
docEdub Dec 5, 2024
1e5b4e1
Remove completed todo
docEdub Dec 5, 2024
c95b4ac
Get sound URLs containing a # working
docEdub Dec 5, 2024
c957e15
Add audio engine `resumeOnPause` options
docEdub Dec 5, 2024
88a2e5d
Handle streaming sound play failures when another user gesture is needed
docEdub Dec 5, 2024
54ec2f1
Fix doc comment
docEdub Dec 5, 2024
cb21e9d
Make `AudioEngineV2.audioContext` a variable and drop offline context
docEdub Dec 5, 2024
b262503
Preload streaming sound instance before handling autoplay option
docEdub Dec 5, 2024
d52cc36
Implement sound `currentTime` getter
docEdub Dec 6, 2024
8a511cb
Remove streaming sound play and stop `waitTime` parameters
docEdub Dec 6, 2024
37b91bf
Implement sound `currentTime` setter
docEdub Dec 6, 2024
1a6b132
Remove jsdoc comments from internal `_StreamingSoundInstance` class
docEdub Dec 6, 2024
a6d8998
Fix umd build error on `Set` iterator
docEdub Dec 6, 2024
75a41ce
Fix `WebAudioEngine.resume` so it returns promise in all cases
docEdub Dec 11, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions packages/dev/core/src/Audio/index.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
/* eslint-disable import/no-internal-modules */
export * from "./Interfaces/IAudioEngine";
export * from "./Interfaces/ISoundOptions";
export * from "./v2/index";
export * from "./analyser";
export * from "./audioEngine";
export * from "./audioSceneComponent";
Expand Down
30 changes: 30 additions & 0 deletions packages/dev/core/src/Audio/v2/abstractAudioBus.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
import type { Nullable } from "../../types";
import type { AudioEngineV2 } from "./audioEngineV2";
import { AbstractNamedAudioNode, AudioNodeType } from "./abstractAudioNode";

/**
* Options for creating a new audio bus node.
*/
export interface IAbstractAudioBusOptions {
/**
* The volume of the audio bus.
*/
volume?: number;
}

/**
* Abstract class representing an audio bus node with a volume control.
*/
export abstract class AbstractAudioBus extends AbstractNamedAudioNode {
/**
* The volume of the audio bus.
*/
public volume: number;

/** @internal */
constructor(name: string, engine: AudioEngineV2, options: Nullable<IAbstractAudioBusOptions> = null) {
super(name, engine, AudioNodeType.InputOutput);

this.volume = options?.volume ?? 1;
}
}
210 changes: 210 additions & 0 deletions packages/dev/core/src/Audio/v2/abstractAudioNode.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,210 @@
import { Observable } from "../../Misc/observable";
import type { Nullable } from "../../types";
import type { AudioEngineV2 } from "./audioEngineV2";
import { AbstractAudioNodeParent } from "./abstractAudioNodeParent";

export enum AudioNodeType {
/**
* Input nodes receive audio data from an upstream node.
*/
Input = 1,

/**
* Output nodes send audio data to a downstream node.
*/
Output = 2,

/**
* Input/Output nodes receive audio data from an upstream node and send audio data to a downstream node.
*/
InputOutput = 3,
}

/**
* Abstract class for an audio node.
*/
export abstract class AbstractAudioNode extends AbstractAudioNodeParent {
// If parent is null, node is owned by audio engine.
private _parent: Nullable<AbstractAudioNodeParent> = null;

/**
* The connected downstream audio nodes.
*
* Undefined for input nodes.
*/
protected readonly _connectedDownstreamNodes?: Set<AbstractAudioNode> | undefined;

/**
* The connected upstream audio nodes.
*
* Undefined for output nodes.
*/
protected readonly _connectedUpstreamNodes?: Set<AbstractAudioNode> | undefined;

/**
* The audio engine this node belongs to.
*/
public readonly engine: AudioEngineV2;

/**
* Observable for when the audio node is disposed.
*/
public readonly onDisposeObservable = new Observable<AbstractAudioNode>();

/** @internal */
constructor(engine: AudioEngineV2, nodeType: AudioNodeType, parent: Nullable<AbstractAudioNodeParent> = null) {
super();

this.engine = engine;
this.parent = parent;

if (nodeType | AudioNodeType.Input) {
this._connectedDownstreamNodes = new Set<AbstractAudioNode>();
}

if (nodeType | AudioNodeType.Output) {
this._connectedUpstreamNodes = new Set<AbstractAudioNode>();
}
}

/**
* Releases associated resources.
*/
public override dispose(): void {
super.dispose();

this.parent.children.delete(this);

if (this._connectedDownstreamNodes) {
for (const node of Array.from(this._connectedDownstreamNodes)) {
this._disconnect(node);
}
this._connectedDownstreamNodes.clear();
}

if (this._connectedUpstreamNodes) {
for (const node of Array.from(this._connectedUpstreamNodes)) {
node._disconnect(this);
}
this._connectedUpstreamNodes.clear();
}

this.onDisposeObservable.notifyObservers(this);
this.onDisposeObservable.clear();
}

/**
* The parent audio node.
*/
public get parent(): AbstractAudioNodeParent {
return this._parent ?? this.engine;
}

/**
* Sets the parent audio node.
*/
public set parent(parent: Nullable<AbstractAudioNodeParent>) {
if (this._parent === parent) {
return;
}

this.parent.children.delete(this);
this._parent = parent;
this.parent.children.add(this);
}

/**
* The audio node's type.
*/
public get type(): AudioNodeType {
let type = 0;

if (this._connectedDownstreamNodes) {
type |= AudioNodeType.Output;
}

if (this._connectedUpstreamNodes) {
type |= AudioNodeType.Input;
}

return type;
}

/**
* Gets a string identifying the name of the class
* @returns the class's name as a string
*/
public abstract getClassName(): string;

/**
* Connect to a downstream audio input node.
* @param node - The downstream audio input node to connect
*/
protected _connect(node: AbstractAudioNode): void {
if (!this._connectedDownstreamNodes) {
return;
}

if (this._connectedDownstreamNodes.has(node)) {
return;
}

if (!node._onConnect(this)) {
return;
}

this._connectedDownstreamNodes.add(node);
}

/**
* Disconnect from a downstream audio input node.
* @param node - The downstream audio input node to disconnect
*/
protected _disconnect(node: AbstractAudioNode): void {
if (!this._connectedDownstreamNodes) {
return;
}

this._connectedDownstreamNodes.delete(node);

node._onDisconnect(this);
}

/**
* Called when an upstream audio output node is connecting.
* @param node - The connecting upstream audio node
* @returns `true` if the connection succeeds; otherwise `false`
*/
protected _onConnect(node: AbstractAudioNode): boolean {
if (!this._connectedUpstreamNodes) {
return false;
}

this._connectedUpstreamNodes.add(node);

return true;
}

/**
* Called when an upstream audio output node disconnects.
* @param node - The disconnecting upstream audio node
*/
protected _onDisconnect(node: AbstractAudioNode): void {
this._connectedUpstreamNodes?.delete(node);
}
}

/**
* Abstract class for an audio node with a name.
*/
export abstract class AbstractNamedAudioNode extends AbstractAudioNode {
/**
* The name of the audio node.
*/
public name: string;

constructor(name: string, engine: AudioEngineV2, nodeType: AudioNodeType) {
super(engine, nodeType);
this.name = name;
}
}
24 changes: 24 additions & 0 deletions packages/dev/core/src/Audio/v2/abstractAudioNodeParent.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import type { IDisposable } from "../../scene";
import type { AbstractAudioNode } from "./abstractAudioNode";

/**
* Abstract base class for audio node parents.
*/
export class AbstractAudioNodeParent implements IDisposable {
/**
* The children audio nodes.
*/
public readonly children = new Set<AbstractAudioNode>();

/**
* Releases associated resources.
*/
public dispose(): void {
if (this.children) {
for (const node of Array.from(this.children)) {
node.dispose();
}
this.children.clear();
}
}
}
Loading
Loading