Clarify Google Vertex AI API schema usage - #128648
Conversation
|
@brendonmatheson : Thanks for your contribution! The author(s) and reviewer(s) have been notified to review your proposed change. @dlepow |
|
@brendonmatheson : Thanks for your contribution! The author(s) and reviewer(s) have been notified to review your proposed change. @dlepow |
|
Learn Build status updates of commit 89b8d6e: ✅ Validation status: passed
For more details, please refer to the build report. |
|
Can you review the proposed changes? IMPORTANT: When the changes are ready for publication, adding a #label:"aq-pr-triaged" |
|
My proposed change should be verified with the product team - I've made the proposal based on a day of testing and searching but I'm blackbox testing this since I cannot find any docs that show how to get generateContent to work. The go-to AI Gateway sample project uses the Open AI schema rather than generateContent: https://github.com/Azure-Samples/AI-Gateway/tree/main/labs/google-gemini-api And I have been unable to find any samples or docs that show llm-emit-token-metrics grabbing token counts from the usageMetadata element of a generateContent response. But the absence of such docs doesnt necessarily mean the feature doesn't exist. |
|
Adding @anishtallapureddy. Anish, could you comment on the change? Thanks. |
|
#assign: @anishtallapureddy |
Google support both their native generateDontent API and the OpenAI compatible API on their platform.
The mention of Google Vertex AI API leads one to believe it's the native format, not the OpenAI compatible format. But from my testing and scouring the internet, we do not support the generateContent API schema, only the OpenAI schema for Google Gemini models.
Therefore, I'm proposing to add the clarification so that readers are not confused and don't waste time trying to get generateContent working.