Repository navigation
feat: agent reply wrapper in component client - #1318
Conversation
|
Ok, this one is ready for review. |
patriknw
left a comment
There was a problem hiding this comment.
looking good, but missing docs
|
|
||
| private def toTokenUsage(metadata: Metadata): Agent.TokenUsage = { | ||
| val input = metadata.get("input_tokens").map[Integer](_.toInt).orElse(0) | ||
| val output = metadata.get("output_tokens").map[Integer](_.toInt).orElse(0) |
There was a problem hiding this comment.
shall we define those metadata keys in the spi to make sure runtime and sdk use the same constants
There was a problem hiding this comment.
yeap we could, should we also add some prefix, spi-input-tokens, spi-agent-input-tokens, agent-input-tokens?
There was a problem hiding this comment.
yes, probably good to avoid possible name clash with user metadata
|
docs added, and here is matching PR in the runtime: https://github.com/lightbend/akka-runtime/pull/4762/changes |
octonato
left a comment
There was a problem hiding this comment.
Looks good, but we still have a type issue to solve.
See my comment below.
| override def withMetadata(metadata: Metadata): ComponentMethodRefImpl[A1, R] = | ||
| componentMethodRefImpl.withMetadata(metadata) | ||
|
|
||
| override def withRetry(retrySettings: RetrySettings): ComponentMethodRefImpl[A1, R] = | ||
| componentMethodRefImpl.withRetry(retrySettings) | ||
|
|
||
| override def withRetry(maxRetries: Int): ComponentMethodRefImpl[A1, R] = componentMethodRefImpl.withRetry(maxRetries) |
There was a problem hiding this comment.
We will need some more type massage here.
It won't be possible to call any of those methods and then withDetailedReply
componentClient
.forAgent()
.inSession(newSessionId())
.method(SomeAgent::mapLlmResponse)
.withRetry(2) // returns ComponentMethodRefImpl
.withDetailedReply() // non-existent method
.invoke("hello");There was a problem hiding this comment.
right, that complicates things, but I think now I support it
No description provided.