Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename rtvi-client-android to pipecat-client-android #7

Merged
merged 5 commits into from
Dec 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -37,13 +37,13 @@ jobs:
java-version: '17'

- name: Build core SDK
run: ./gradlew :rtvi-client-android:assembleRelease
run: ./gradlew :pipecat-client-android:assembleRelease

- name: Generate Maven repo
run: ./gradlew :rtvi-client-android:publishAllPublicationsToRTVILocalRepoRepository
run: ./gradlew :pipecat-client-android:publishAllPublicationsToPipecatLocalRepoRepository

- name: Upload Maven repo
uses: actions/upload-artifact@v4
with:
name: RTVIClientAndroid-Repo
path: build/RTVILocalRepo
name: PipecatClientAndroid-Repo
path: build/PipecatLocalRepo
6 changes: 5 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
# 0.3.0

- Project renamed to `pipecat-client-android`

# 0.2.1

- Added callbacks:
Expand Down Expand Up @@ -30,4 +34,4 @@
- Added callbacks
- `onBotLLMText()`
- `onBotTTSText()`
- `onStorageItemStored()`
- `onStorageItemStored()`
16 changes: 10 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,25 +1,29 @@
# Real-Time Voice Inference Android Client SDK
<h1><div align="center">
<img alt="pipecat" width="500px" height="auto" src="assets/pipecat-android.png">
</div></h1>

# Pipecat Android Client SDK

[RTVI](https://github.com/rtvi-ai/) is an open standard for Real-Time Voice (and Video) Inference.

This Android library contains the core components and types needed to set up an RTVI session.

When building an RTVI application, you should use the transport-specific client library (see
[here](https://rtvi.mintlify.app/api-reference/transports/introduction) for available first-party
packages.) The base `VoiceClient` has no transport included.
packages.) The base `RTVIClient` has no transport included.

## Usage

Add the following dependency to your `build.gradle` file:

```
implementation "ai.rtvi:client:0.2.1"
implementation "ai.pipecat:client:0.3.0"
```

Then instantiate the `VoiceClient` from your code, specifying the backend `baseUrl` and transport.
Then instantiate the `RTVIClient` from your code, specifying the backend `baseUrl` and transport.

```kotlin
val callbacks = object : VoiceEventCallbacks() {
val callbacks = object : RTVIEventCallbacks() {

override fun onBackendError(message: String) {
Log.e(TAG, "Error from backend: $message")
Expand All @@ -28,7 +32,7 @@ val callbacks = object : VoiceEventCallbacks() {
// ...
}

val client = VoiceClient(baseUrl, transport, callbacks)
val client = RTVIClient(transport, callbacks, options)

client.start().withCallback {
// ...
Expand Down
Binary file added assets/pipecat-android.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ plugins {
}

android {
namespace = "ai.rtvi.client"
namespace = "ai.pipecat.client"
compileSdk = 34

defaultConfig {
Expand Down Expand Up @@ -51,40 +51,40 @@ dependencies {
publishing {
repositories {
maven {
url = rootProject.layout.buildDirectory.dir("RTVILocalRepo").get().asFile.toURI()
name = "RTVILocalRepo"
url = rootProject.layout.buildDirectory.dir("PipecatLocalRepo").get().asFile.toURI()
name = "PipecatLocalRepo"
}
}

publications {
register<MavenPublication>("release") {
groupId = "ai.rtvi"
groupId = "ai.pipecat"
artifactId = "client"
version = "0.2.1"
version = "0.3.0"

pom {
name.set("RTVI Client")
description.set("Core RTVI client library for Android")
url.set("https://github.com/rtvi-ai/rtvi-client-android")
name.set("Pipecat Client")
description.set("Core Pipecat client library for Android")
url.set("https://github.com/pipecat-ai/pipecat-client-android")

developers {
developer {
id.set("rtvi.ai")
name.set("rtvi.ai")
id.set("pipecat.ai")
name.set("pipecat.ai")
}
}

licenses {
license {
name.set("BSD 2-Clause License")
url.set("https://github.com/rtvi-ai/rtvi-client-android/blob/main/LICENSE")
url.set("https://github.com/pipecat-ai/pipecat-client-android/blob/main/LICENSE")
}
}

scm {
connection.set("scm:git:git://github.com/rtvi-ai/rtvi-client-android.git")
developerConnection.set("scm:git:ssh://github.com:rtvi-ai/rtvi-client-android.git")
url.set("https://github.com/rtvi-ai/rtvi-client-android")
connection.set("scm:git:git://github.com/pipecat-ai/pipecat-client-android.git")
developerConnection.set("scm:git:ssh://github.com:pipecat-ai/pipecat-client-android.git")
url.set("https://github.com/pipecat-ai/pipecat-client-android")
}
}

Expand All @@ -103,4 +103,4 @@ signing {
useInMemoryPgpKeys(signingKey, signingPassphrase)
sign(publishing.publications)
}
}
}
Original file line number Diff line number Diff line change
@@ -1,37 +1,37 @@
package ai.rtvi.client

import ai.rtvi.client.helper.RTVIClientHelper
import ai.rtvi.client.helper.RegisteredRTVIClient
import ai.rtvi.client.result.Future
import ai.rtvi.client.result.Promise
import ai.rtvi.client.result.RTVIError
import ai.rtvi.client.result.RTVIException
import ai.rtvi.client.result.Result
import ai.rtvi.client.result.resolvedPromiseErr
import ai.rtvi.client.result.withPromise
import ai.rtvi.client.result.withTimeout
import ai.rtvi.client.transport.AuthBundle
import ai.rtvi.client.transport.MsgClientToServer
import ai.rtvi.client.transport.MsgServerToClient
import ai.rtvi.client.transport.Transport
import ai.rtvi.client.transport.TransportContext
import ai.rtvi.client.transport.TransportFactory
import ai.rtvi.client.types.ActionDescription
import ai.rtvi.client.types.Config
import ai.rtvi.client.types.MediaDeviceId
import ai.rtvi.client.types.Option
import ai.rtvi.client.types.RegisteredHelper
import ai.rtvi.client.types.ServiceConfig
import ai.rtvi.client.types.ServiceConfigDescription
import ai.rtvi.client.types.Transcript
import ai.rtvi.client.types.TransportState
import ai.rtvi.client.types.Value
import ai.rtvi.client.utils.ConnectionBundle
import ai.rtvi.client.utils.JSON_INSTANCE
import ai.rtvi.client.utils.ThreadRef
import ai.rtvi.client.utils.parseServerSentEvents
import ai.rtvi.client.utils.post
import ai.rtvi.client.utils.valueFrom
package ai.pipecat.client

import ai.pipecat.client.helper.RTVIClientHelper
import ai.pipecat.client.helper.RegisteredRTVIClient
import ai.pipecat.client.result.Future
import ai.pipecat.client.result.Promise
import ai.pipecat.client.result.RTVIError
import ai.pipecat.client.result.RTVIException
import ai.pipecat.client.result.Result
import ai.pipecat.client.result.resolvedPromiseErr
import ai.pipecat.client.result.withPromise
import ai.pipecat.client.result.withTimeout
import ai.pipecat.client.transport.AuthBundle
import ai.pipecat.client.transport.MsgClientToServer
import ai.pipecat.client.transport.MsgServerToClient
import ai.pipecat.client.transport.Transport
import ai.pipecat.client.transport.TransportContext
import ai.pipecat.client.transport.TransportFactory
import ai.pipecat.client.types.ActionDescription
import ai.pipecat.client.types.Config
import ai.pipecat.client.types.MediaDeviceId
import ai.pipecat.client.types.Option
import ai.pipecat.client.types.RegisteredHelper
import ai.pipecat.client.types.ServiceConfig
import ai.pipecat.client.types.ServiceConfigDescription
import ai.pipecat.client.types.Transcript
import ai.pipecat.client.types.TransportState
import ai.pipecat.client.types.Value
import ai.pipecat.client.utils.ConnectionBundle
import ai.pipecat.client.utils.JSON_INSTANCE
import ai.pipecat.client.utils.ThreadRef
import ai.pipecat.client.utils.parseServerSentEvents
import ai.pipecat.client.utils.post
import ai.pipecat.client.utils.valueFrom
import android.util.Log
import kotlinx.serialization.json.JsonElement
import kotlinx.serialization.json.JsonPrimitive
Expand All @@ -40,7 +40,7 @@ import kotlinx.serialization.json.jsonObject
import okhttp3.MediaType.Companion.toMediaType
import okhttp3.RequestBody.Companion.toRequestBody

private const val RTVI_PROTOCOL_VERSION = "0.2.0"
private const val RTVI_PROTOCOL_VERSION = "0.3.0"

/**
* An RTVI client. Connects to an RTVI backend and handles bidirectional audio and video
Expand All @@ -59,7 +59,7 @@ open class RTVIClient(
private var options: RTVIClientOptions,
) {
companion object {
private const val TAG = "VoiceClient"
private const val TAG = "RTVIClient"
}

/**
Expand Down Expand Up @@ -252,9 +252,6 @@ open class RTVIClient(
*/
fun initDevices(): Future<Unit, RTVIError> = transport.initDevices()

@Deprecated("start() renamed to connect()")
fun start() = connect()

/**
* Initiate an RTVI session, connecting to the backend.
*/
Expand All @@ -267,42 +264,51 @@ open class RTVIClient(
)
}

transport.setState(TransportState.Authorizing)
if (options.params.baseUrl != null && options.params.endpoints.connect != null) {

// Send POST request to the provided base_url to connect and start the bot
transport.setState(TransportState.Authorizing)

val connectionData = ConnectionData.from(options)
// Send POST request to the provided base_url to connect and start the bot

val body = ConnectionBundle(
services = options.services?.associate { it.service to it.value },
config = connectionData.config
)
.serializeWithCustomParams(connectionData.requestData)
.toRequestBody("application/json".toMediaType())
val connectionData = ConnectionData.from(options)

val currentConnection = Connection().apply { connection = this }
val body = ConnectionBundle(
services = options.services?.associate { it.service to it.value },
config = connectionData.config
)
.serializeWithCustomParams(connectionData.requestData)
.toRequestBody("application/json".toMediaType())

return@runOnThreadReturningFuture post(
thread = thread,
url = options.params.baseUrl + options.params.endpoints.connect,
body = body,
customHeaders = connectionData.headers
)
.mapError<RTVIError> {
RTVIError.HttpError(it)
}
.chain { authBundle ->
if (currentConnection == connection) {
transport.connect(AuthBundle(authBundle))
} else {
resolvedPromiseErr(thread, RTVIError.OperationCancelled)
val currentConnection = Connection().apply { connection = this }

return@runOnThreadReturningFuture post(
thread = thread,
url = options.params.baseUrl + options.params.endpoints.connect,
body = body,
customHeaders = connectionData.headers
)
.mapError<RTVIError> {
RTVIError.HttpError(it)
}
}
.chain { currentConnection.ready }
.withTimeout(30000)
.withErrorCallback {
disconnect()
}
.chain { authBundle ->
if (currentConnection == connection) {
transport.connect(AuthBundle(authBundle))
} else {
resolvedPromiseErr(thread, RTVIError.OperationCancelled)
}
}
.chain { currentConnection.ready }
.withTimeout(30000)
.withErrorCallback {
disconnect()
}

} else {
// No connection endpoint
Log.w(TAG, "No connect endpoint specified, skipping auth request")
connection = Connection()
return@runOnThreadReturningFuture transport.connect(null)
}
}

/**
Expand Down Expand Up @@ -597,11 +603,10 @@ private class ConnectionData(
) {
companion object {
fun from(value: RTVIClientOptions) = ConnectionData(
headers = value.customHeaders + value.params.headers,
headers = value.params.headers,
requestData = listOf("rtvi_client_version" to Value.Str(RTVI_PROTOCOL_VERSION))
+ value.customBodyParams
+ value.params.requestData,
config = value.config + value.params.config
config = value.params.config
)
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
package ai.pipecat.client

import ai.pipecat.client.types.ServiceConfig
import ai.pipecat.client.types.ServiceRegistration
import ai.pipecat.client.types.Value

/**
* Configuration options when instantiating a [RTVIClient].
*/
data class RTVIClientOptions(

/**
* Connection parameters.
*/
val params: RTVIClientParams,

/**
* Enable the user mic input.
*
* Defaults to true.
*/
val enableMic: Boolean = true,

/**
* Enable user cam input.
*
* Defaults to false.
*/
val enableCam: Boolean = false,

/**
* A list of services to use on the backend.
*/
val services: List<ServiceRegistration>? = null,
)
Loading
Loading