Blog
how to reduce aws costsreduce gpu inference costsaws cost optimization

How to Reduce AWS Costs with AI-Assisted Cost Reviews

A step-by-step method for cutting your AWS bill: export billing data, use AI to find waste, verify it, and fix GPU inference costs. Prompts included.

Portrait of Luis Chavez-Mattos
Edited by Luis Chavez-Mattos, Director, Product··RSS

For Engineering and Executives

A person holds an oversized printed cloud billing invoice up to the light, squinting at a single highlighted line item.

How do you reduce AWS costs with an AI-assisted review?

Export your billing data (Cost Explorer or the Cost and Usage Report), give an AI model a summarized version of it, and ask it to rank your spend by service, flag likely waste, and propose specific fixes. Then verify every finding in the AWS console before changing anything. The AI is good at reading thousands of line items and spotting patterns. It is bad at knowing which resource is safe to delete. You supply that judgment, ideally with the engineer who owns the resource.

This post gives you the full process: the data to export, the prompts to paste, the checks to run on the answers, and a separate section on cutting GPU inference costs, which behave differently from the rest of your bill.

TL;DR

  • Start with the bill, not the architecture. Most savings come from a short list of line items, so rank spend by service and usage type before you look at any individual resource.
  • Use AI to read and sort, not to decide. It summarizes large exports well and hypothesizes about causes, but a human confirms each finding in the console.
  • Fix the boring waste first. Idle resources, oversized instances, old storage, excess log retention and data transfer charges usually come before any redesign.
  • Commit to discounts last. Savings Plans and Reserved Instances only pay off after you have right-sized, or you lock in waste.
  • GPU inference is a utilization problem. Cost falls when GPUs stay busy: smaller or quantized models, batching, autoscaling, and the right instance type.
  • Make it a monthly routine with owners. A one-time cleanup decays; a tracked list of findings with names and dates does not.

What data do you need before an AI cost review?

You need three things. Gather them before you open any AI tool.

  1. A cost breakdown by service and usage type. In AWS Cost Explorer, group by Service, then by Usage Type, for the last three to six months. Export to CSV. This shows where the money goes and how it trends.
  2. A breakdown by tag or account. Group by linked account, and by a cost allocation tag such as team or environment if you have one. If you have no tags, that is itself your first finding.
  3. Utilization data for your biggest compute and database items. Compute Optimizer and Trusted Advisor (availability of some checks depends on your support plan) give right-sizing and idle-resource recommendations. CloudWatch gives CPU, memory (with the agent installed), network and, for GPUs, utilization metrics.

If your company uses the full Cost and Usage Report (CUR), it is the most detailed source, but the file is large. Do not paste raw CUR into a chat window. Aggregate it first, with a query in Athena or a spreadsheet pivot, into a table of a few hundred rows at most.

Data handling: Billing exports can include account IDs, resource names and internal project names. Check your company's policy on which AI tools are approved for internal data. Remove account IDs and anything sensitive. Never paste credentials, access keys or full configuration files.

How do you run the review step by step?

Step 1: Rank the spend

Paste your summarized export (service, usage type, monthly cost for each month) and use this prompt:

You are a cloud cost analyst. Below is a CSV of our AWS spend by service and usage type for the last [N] months. 

1. Rank the top 10 cost drivers by total spend and show the month-over-month trend for each.
2. Flag any line item that grew faster than the rest, and any that is unusually large for its category (for example NAT gateway data processing, inter-region or internet data transfer, CloudWatch Logs ingestion, idle-looking load balancers).
3. For each flagged item, list 2-3 plausible causes and the exact place in the AWS console or CLI where I can verify each one.
4. Do not recommend deleting anything. Only tell me what to investigate.

Data:
[paste CSV]

The last instruction matters. It keeps the model in the role it is good at: reading and hypothesizing.

Step 2: Check the output against the data

Before you act on anything, run these checks on the AI's answer:

  • Do the numbers add up? Spot-check three figures against your own export. Models sometimes misread a column or sum the wrong rows.
  • Is every claim traceable? Each finding should point to a line in your data. If it says "your NAT gateway costs are high" and you cannot find that line, discard it.
  • Are the causes plausible for your setup? A model does not know that a spike was a planned migration. Ask the team.

Step 3: Investigate the flagged items

Take the top five findings and verify each in the console or with the owning engineer. The table below lists where waste commonly hides and how to confirm it.

Common source of wasteHow to confirmTypical fix
Idle or oversized EC2 instancesCompute Optimizer, CloudWatch CPU and memory over several weeksDownsize, change instance family, or stop
Unattached EBS volumes and old snapshotsEC2 console, volumes in "available" state; snapshot ageDelete after owner sign-off, set snapshot lifecycle
Older volume types (gp2)Volume type columnMigrate to gp3, usually a low-risk change
NAT gateway data processingCost Explorer usage type, VPC flow logsAdd VPC endpoints for S3 and DynamoDB, reduce cross-AZ traffic
S3 storage in the wrong classS3 Storage Lens, bucket size by classLifecycle rules, Intelligent-Tiering, delete incomplete multipart uploads
CloudWatch Logs with no retentionLog groups set to "Never expire"Set retention, lower log verbosity
Idle load balancers and Elastic IPsRequest counts near zero; unattached IPsRemove after confirming
Dev and test running 24/7Instance schedules, tagsSchedule shutdown nights and weekends
RDS instances oversized or multi-AZ in devRDS metrics, configRight-size, single-AZ for non-production
Data transfer between regions or to the internetUsage types containing "DataTransfer"Move workloads closer, use a CDN, compress

Step 4: Ask AI for a prioritized action plan

Once you have verified findings, give the model the facts you confirmed and ask for a plan:

Here are the findings I have verified in our AWS environment, each with monthly cost, the owner, and the risk of changing it:

[paste your table: finding, monthly cost, owner, risk notes]

Create a prioritized action plan. Order by estimated monthly savings divided by effort and risk. For each action give:
- the specific change
- how to roll it back
- who needs to approve it
- how we will confirm savings on next month's bill

Mark anything where you are guessing at savings. Do not invent prices; tell me where to look up current pricing instead.

That last line is deliberate. Models often produce confident-sounding price figures that are out of date. Always check current rates on the AWS pricing pages or the AWS Pricing Calculator.

Step 5: Right-size first, then buy commitments

Savings Plans and Reserved Instances lower the hourly rate in exchange for a commitment of one or three years. If you buy them before cleaning up, you commit to paying for capacity you should have removed. The order is:

  1. Delete idle resources.
  2. Right-size what remains.
  3. Move suitable workloads to cheaper options (Graviton-based instances where your software supports ARM, Spot for fault-tolerant jobs).
  4. Then cover your steady baseline with a commitment, and leave a margin below your minimum usage.

Ask the AI to help model scenarios, but have finance and engineering agree on the baseline. A prompt that works: "Given this hourly compute usage profile, show the minimum steady usage over the last 90 days and what share of usage that would cover under a commitment. List the risks if our usage drops by a third."

Step 6: Track results and repeat monthly

Savings that are not measured do not exist. Keep a simple log: finding, owner, action, date, expected monthly savings, actual savings on the next bill. Review it once a month, and rerun Step 1 with the new export. New waste appears constantly, usually from new projects.

How do you reduce GPU inference costs on AWS?

GPU instances are among the most expensive things on an AWS bill, and the main driver of cost is not the hourly price. It is how much of that hour the GPU does useful work. A GPU sitting at low utilization costs the same as one running flat out. Work through these in order.

1. Measure real utilization

Collect GPU utilization, GPU memory use, request rate and latency per endpoint over at least a few weeks. Ask the AI to analyze the exported metrics:

Below are hourly metrics for our GPU inference endpoints: instance type, GPU utilization %, GPU memory used, requests per second, p95 latency, and hourly cost.

1. Identify endpoints with persistently low GPU utilization.
2. Identify hours of the day or days of the week with low traffic where capacity could scale down.
3. Tell me whether memory or compute looks like the limiting factor for each endpoint.
4. List questions I should ask the team before changing anything.

Data:
[paste]

2. Use a smaller or quantized model

Many production tasks (classification, extraction, summarization, routing) do not need the largest model. A smaller model, or the same model quantized to lower numeric precision, needs less GPU memory and often runs faster, which can mean a cheaper instance or more requests per GPU. The tradeoff is quality. Test on your own examples, not on public benchmarks, and set an acceptable quality bar before you start.

3. Batch requests and use an efficient serving stack

GPUs are much more efficient when they process several requests together. Modern inference servers support continuous or dynamic batching, and this often raises throughput substantially without new hardware. If your serving setup handles one request at a time, this is likely your largest win. Check that latency still meets your requirement after batching.

4. Scale to demand

If traffic is uneven, autoscale on request load or queue depth rather than keeping peak capacity running all day. For work that is not interactive (document processing, nightly enrichment), use asynchronous or batch patterns so jobs queue and run on fewer GPUs. For rarely used models, consider serverless or scale-to-zero options, accepting cold-start delays.

5. Choose the right instance and purchase option

  • Match the GPU to the model. Pick the cheapest instance whose GPU memory fits the model and meets your latency goal. An older or smaller GPU is often enough for inference, unlike training.
  • Consider AWS's own inference chips. AWS offers Inferentia-based instances for inference. They can lower cost per request for supported models, but require compiling your model with the AWS Neuron SDK, so test compatibility and effort first.
  • Use Spot for fault-tolerant work. Spot capacity is cheaper but can be reclaimed. It suits batch jobs and stateless replicas behind a load balancer, not a single production endpoint.
  • Commit only to the steady baseline. Same rule as before: right-size, then commit.

6. Cache and route

Many inference requests repeat. Caching responses for identical or near-identical inputs avoids GPU work entirely. For applications that mix easy and hard requests, route simple ones to a small model and send only hard ones to a large one.

7. Compare self-hosting with a managed service

If you run a model yourself mainly because it was the first thing that worked, compare the total cost against a managed option such as Amazon Bedrock or another hosted API, which charges per use rather than per hour. Self-hosting tends to win at high, steady volume. Managed pricing tends to win at low or spiky volume, since you do not pay for idle GPUs. Do this comparison with your actual request volumes and current published prices, and include the engineering time to operate the GPUs.

Where should you not trust AI in a cost review?

  • Prices and discount rates. AWS prices vary by region and change. Use the pricing pages and Pricing Calculator for any number that goes into a business case.
  • "Safe to delete" judgments. The model sees a bill, not your dependencies. A volume that looks orphaned may be a backup someone needs next quarter.
  • Savings estimates. Treat them as hypotheses. The real figure is what shows up on next month's bill.
  • Anything involving production changes. Use change management, a rollback plan and the resource owner's approval.
  • Sensitive data. Follow your company's rules on what can be pasted into which tool.

Who should be involved in the review?

A good review needs three perspectives. An engineer or architect who knows what each workload does and can say what is safe to change. A finance or FinOps partner who understands commitments, budgets and chargeback. An executive sponsor who can settle tradeoffs, such as accepting slightly higher latency for a lower bill, and who makes sure findings get owners. Without the sponsor, review lists tend to stall because no team has time to act.

Can you build your own cost-review tracker instead of using spreadsheets?

Most teams end up with a spreadsheet of findings, owners and savings that goes stale within a quarter. If you want something that fits your process, you can describe it to Remy in plain English: a small internal app where reviewers log each finding, assign an owner, set a status, record expected and actual monthly savings, and see totals by team. Remy turns that description into a spec and a deployed full-stack app with sign-in, a database and a live URL, so engineers and finance can both use it. Because you own the code, you can adjust the fields as your review process changes, and you are not adding another per-seat subscription.

Frequently Asked Questions

5 questions

How often should we do an AWS cost review?

Monthly for the top-level review, because bills close monthly and new waste appears constantly. Add alerts for anomalies (AWS Budgets and Cost Anomaly Detection) so large surprises are caught within days. Do a deeper review, including commitments, quarterly.

What is the quickest way to cut an AWS bill?

Start with idle and oversized resources, unattached storage, log retention and non-production environments left running around the clock. These typically need little engineering effort and carry low risk. Redesigns and commitments come after.

Can AI automatically reduce my AWS costs?

AI can analyze exports, find patterns and draft action plans, and some tools can apply changes automatically. For most companies, the safer approach is AI for analysis and humans for approval and execution. An automated delete based on a wrong guess costs more than the savings.

Why is my GPU inference bill so high?

Usually because GPUs run at low utilization, the model is larger than the task needs, requests are not batched, or capacity is sized for peak traffic all day. Measure utilization first. Then apply a smaller or quantized model, batching, autoscaling and the right instance type, in that order.

Is it safe to paste my AWS billing data into an AI tool?

Only if your company approves the tool for internal data. Aggregate the data, remove account IDs and resource names that reveal sensitive projects, and never include credentials or access keys. When in doubt, ask your security team.

How we source, write, and correct our articles. Editorial standards

Related articles

Straight to your inbox.
New pieces as they publish. No noise.