LinkedIn
Twitter
Yesterday i was testing Vertex AI fine tuning with using text-bison model a 3KB file (10 examples total). The first try took about 3 hours and failed at the end while creating an endpoint, the very last step.
I tried it agiain with Compute Engine API service account another time which took around 2,3 hours and succeded. This was also with the same .jsonl file of 3 KB in size.
When i check this morning i have noticed that i was charged $254 for this.
I would like to know why? How is this possbile?
Considering I ll have 14000+ examples on my original data, i would like to know more about htis pricing becuase this does being stated anywhere....
Ok here is the problem... When you setup your fine tuning with text-bison model, the accelerator type is automatically chosen as shown below.
Here are the two definitions of the accelerators:
NVIDIA A100 80GB GPU
- Most expensive option. The hourly cost can range significantly depending on the region, but it's generally considered the premium choice for heavy-duty workloads.
- Example: In the us-central1 region, the cost could be around $3 per hour or more.
TPU 64 Core v3 Pod
- Significantly more cost-effective than A100 GPUs. TPUs are designed to offer better price-to-performance for certain machine learning workflows.
- Complexity: Pricing for TPUs is sometimes listed per pod (which contains multiple TPU chips), making direct hourly comparisons tricky.
- Example: In the us-central1 region, a v3-8 TPU (which is a smaller unit) costs around $1 per hour. A full pod would likely be proportionally more but still significantly less expensive than a comparable time on the A100.
(Provided by Gemini)
Yet this still does NOT explain why 10 examples took 3 hours adn costs above $200.
