Skip to main content

Get the Reddit app

Scan this QR code to download the app now
Or check it out in the app stores
r/aws icon
r/aws icon

r/aws

members
online

Calling all new AWS users: read this first!
MOD

Hello and welcome to the /r/AWS subreddit! We are here to support those that are new to Amazon Web Services (AWS) along with those that continue to maintain and deploy on the AWS Cloud! An important consideration of utilizing the AWS Cloud is controlling operational expense (costs) when maintaining your AWS resources and services utilized.

We've curated a set of documentation, articles and posts that help to understand costs along with controlling them accordingly. See below for recommended reading based on your AWS journey:

If you're new to AWS and want to ensure you're utilizing the free tier..

If you're a regular user (think: developer / engineer / architect) and want to ensure costs are controlled and reduce/eliminate operational expense surprises..

Enable multi-factor authentication whenever possible!

Continued reading material, straight from the /r/AWS community..

Please note, this is a living thread and we'll do our best to continue to update it with new resources/blog posts/material to help support the community.

Thank you!

Your /r/AWS Moderation Team

changelog
09.09.2023_v1.3 - Readded post
12.31.2022_v1.2 - Added MFA entry and bumped back to the top.
07.12.2022_v1.1 - Revision includes post about MFA, thanks to a /u/fjleon for the reminder!
06.28.2022_v1.0 - Initial draft and stickied post

Need help starting open stack Need help starting open stack
security

So, our third year of engineering has just started, and we have a subject named Cloud and Distributed Systems.

Our professor asked a few questions about who knows about the cloud, what you all know, who has used it, etc. I was the only one who had used AWS EC2 and S3 instances, so the professor was impressed. After the lecture, she asked me if I would like to do a project on the cloud where we can use university machines to create a cloud environment for our class. The environment would have features such as storage, instances like EC2, etc.

She told me to learn about OpenStack software, how it functions, and the system requirements so we could ask the university.

Long story short, can you all suggest what I can do, how I can start, and if this project is scalable?

Also, a small P.S.: I am interested in making my career in cyber/cloud security. How can I integrate a security aspect into this project?


how much are you spending a month to host and deploy your app on aws? how much are you spending a month to host and deploy your app on aws?
database

I've been doing research how cheap or expensive hosting an application on aws can be? I am a cs student working on an application currently with 14 prospects that will need it. To be drop some clues it is just collect a persons name,dob, and crime they have committed and have the users view it. Im not sure if a $100 will do without over engineering it.



GitHub - aws/aws-secretsmanager-agent: The AWS Secrets Manager Agent is a local HTTP service that you can install and use in your compute environments to read secrets from Secrets Manager and cache them in memory.

Lambda not parsing emails with attachments Lambda not parsing emails with attachments
serverless

I have a function that parses emails and send to my backend endpoint, while normal emails without attachments get parsed that ones with attachment does not even trigger lambda function ( Since there are no logs on cloudWatch )

When I receive emails I trigger an SNS and using that SNS notification my lambda parses the content in the email. I read somewhere that SNS can carry only 250KB data and therefore emails with attachments are not triggering my lambda function

I am not able to confirm this. And if this is true how should I handle emails with attachments?



To veteran CloudFront/S3 users, why was it designed like this the first time? To veteran CloudFront/S3 users, why was it designed like this the first time?
discussion

I have an internal company website which we made to only be accessible from certain IPs. We are planning to improve speed by optimizing its infrastructure. However, we were surprised to find out that previous guys put CloudFront in the back (as shown below).

https://preview.redd.it/to-veteran-cloudfront-s3-users-why-was-it-designed-like-v0-kb9so17812cd1.png

Infra was first created in 2018/2019, I think. Was this a correct way in the past? Do you guys think there were any special reasons for this?

We are definitely thinking of putting CloudFront at the front, the bucket and ALB behind it, and limit access using WAF IP set rule.

Any insights would be appreciated. Thanks!



How long does Elastic Beanstalk terminated instances stay visible on the console till its finally deleted? How long does Elastic Beanstalk terminated instances stay visible on the console till its finally deleted?
technical question

Trying to use elastic beanstalk for the first time. I terminated an environment and even deleted the application, but the environment is still visible, due to which I'm not able to create a new environment with that name. Its been 30 min.s already



Developers Who've Moved On From AWS Amplify - What Did You Switch To? Developers Who've Moved On From AWS Amplify - What Did You Switch To?
discussion

Hey Folks!

I've been using AWS Amplify for a while and there are a few things I absolutely love about it:

  • As a developer, I appreciate having my own infrastructure stack that I can easily delete if needed.

  • I love being able to edit both my frontend and backend simultaneously, with Lambda code updating remotely while staying available on the frontend.

However, there are also some things I really dislike, especially with Amplify's Genv2:

  • The forced use of GraphQL with their model, which has poorly documented and insufficient basic authorization principles.

I've been thinking about bypassing Amplify entirely and setting up my application stack using something like React + API Gateway + Lambda + DynamoDB. Ideally, I'd like to maintain a similar development approach where:

  1. Every engineer has their own local environment.

  2. Updates are reflected locally during testing.

  3. It provides the same capabilities I enjoyed with Amplify.

So, I'm curious:

  • Is there a good template or AWS service that matches this use case?

  • Should I continue using Amplify but ignore its unnecessary features and instead define my own constructs to publish via CDK?

  • What alternatives have you guys tried and loved?

Looking forward to your insights and experiences!

Thanks!



Nested JSON file from S3 to Redshift Nested JSON file from S3 to Redshift
technical question

Maybe someone can help with this. A department where I work can only generate data as a json file and they want it in our aws data lake. The json file has hundreds of nested tables. I'm looking to flatten it into many pyspark dataframes in a gluejob and then push the data to redshift tables.

I've only ever pulled csv or parquet files in using glue. Another problem is this department can't even give me the schemas ahead of time so I'm really hoping there's a way to get a crawler working here.

I'm also looking at documentdb as a solution instead. Would this be the path of least resistance? I have no experience with nosql databases. My thought here would be to use this to unpack the file and then use a glue job to push to redshift. Ultimately I'll need to join this data with our other tables to push to power bi for the business users.

Thanks for any insight I can get.


Seeking Guidance for Hosting a RAG Chatbot on AWS with any open 7B model or Mistral-7B-Instruct-v0.2 Seeking Guidance for Hosting a RAG Chatbot on AWS with any open 7B model or Mistral-7B-Instruct-v0.2
ai/ml

Hello there,

I'm planning to host a Retrieval-Augmented Generation (RAG) chatbot on AWS using the Mistral-7B-Instruct-v0.2-AWQ model. I’m looking for guidance on the following:

  • Steps: What are the key steps I need to follow to set this up?

  • Resources: Any articles, tutorials, or documentation that can help me through the process?

  • Videos: Are there any video tutorials that provide a walkthrough for deploying similar models on AWS?

I appreciate any tips or insights you can share. Thanks in advance for your help :)




Not Releasing Mic After Call Not Releasing Mic After Call
technical question

Hi!

Not sure if this is the right place for this post but I figured it was at least a good place to start. I’ll try to keep the rambling to a minimum.

I WFH, my company uses AWS for all calls through the browser. I use a Bluetooth multipoint headset to listen to other things between these calls.

I can’t switch to my 2nd audio source if it’s detected that I’m still in a call on my PC in AWS. Perfectly fine, right? Nope. AWS continues controlling the system’s microphone for minutes past when the call ends.

This means that, for almost 5 minutes after the call has ended, I’m still waiting for AWS to release the microphone/stop using it so I can switch back to my 2nd device, & try to continue what else I was doing before another call rolls in & starts the process over again.

I need to somehow force the call to release(?), but nothing I've tried accomplishes that. I've refreshed the browser pages, completely closed them out & reopened them, even disabled/reenabled microphone access for desktop applications in the Bluetooth settings in Windows (AWS just takes control of the mic back as soon as permissions are reenabled 🙄)

There’s probably not anything I can do that will make a difference, but it’s driving me crazy so I had to ask lol



What the hell permissions do I _actually_ need for to create an S3 batch operation job? What the hell permissions do I _actually_ need for to create an S3 batch operation job?
technical question

EDIT: OK, I always forget about iam:PassRole and (unrelated to this) permissions on service VPC endpoints, and end up frustrated with what's going on. I read the docs page multiple times and the note about iam:PassRole right at the top just slid right off my smooth brain every time... Thanks to u/AcrobaticLime6103

I've got a (working) S3 bucket replication configuration between 2 AWS KMS (SSE-KMS) encrypted buckets with different keys (as they're in different regions, but the same account).

I've added to the IAM policy for the replication role permissions to s3:PutInventoryConfiguration (at bucket level) and s3:InitiateReplication (at object level, i.e. <bucket_arn>/*), because my reading of the docs is that that is necessary in additional to the usual replication permissions for batch jobs to work.

I have a 'human' role/policy which I initially created with just the S3 'job' permissions (Including s3:CreateJob), but in desperation at trying to get this damn thing working currently has s3:* and kms:* on resource *. I have also, although I have no idea if it is necessary, granted sts:AssumeRole to it on the replication role, and confirmed that I can assume the replication role through the AWS console.

Even with all that, when I go through the process of trying to create a batch job in the web console I get to the very end, click 'Create Job', and get the error

Access Denied. You or your AWS admin must update your IAM permissions. Learn more about [Identity and access management in Amazon S3](https://docs.aws.amazon.com/console/s3/accesscontrol)

That linked to page is useless. It's all about bucket and object access to S3 directly, there's nothing about batch jobs.

If I grant my human role * on resource * then I am able to create the job, which proceeds to work exactly as intended to replicate existing objects that were created before the replication policy was applied. I am aware of some other pitfalls I can come across with trying to write the [optional] manifest file or results of the batch job and the batch job role needing additional s3:PutObject etc permissions on whatever bucket those go to, but I've already experienced those and they only come into effect after the batch job has been successfully 'created'. Generally the setup steps that must complete before you can 'run' it fail.

Given my human role already has s3:* and kms:* on *, and I can successfully write objects to both buckets when assuming that role (proving that I have permissions to access both source and destination, in case that was a question), there must be some additional non-S3 and non-KMS permission needed to create batch jobs. But I cannot find this documented anywhere.

What the hell is the minimal permission set to just successfully create a batch job through the web console, regardless of whether the job initialises so it can be run or not...


Storing thumbnail images in s3 Storing thumbnail images in s3
discussion

Hi everyone,

I have a use case wherein I need to store thumbnail images in s3 and store the path in a database for those images. So the UI will have many such thumbnail images so I was wondering what is the best way to store these images as the image binary will be stored that will be rendered on the UI so I am assuming that will be many calls to s3.since it is a thumbnail image I was wondering if I should render it as an image url or store the image in s3?

What do you folks suggest?

Sorry my English is not that good,


Why don't policies for cloudfront signed URLs/cookies have a condition for the HTTP method? Why don't policies for cloudfront signed URLs/cookies have a condition for the HTTP method?
discussion

This seems like such a dumb feature to not include, right? They support wildcards in the url, source IPs, but somehow not something as basic as the http method?

I think I can still accomplish what I need by using Cloudfront Functions and a KeyValueStore, but I'd at least like to understand the reasoning behind this...



How to Clean up Training Account Daily How to Clean up Training Account Daily
general aws

As there isn't that many free aws hands-on training available, we are in the process of setting up a training account and having users login to do stuff in that. My question is, what's the best way to reset the environment each day? Is there an easy way to just delete everything and have it start back from 0? Maybe some type of cloudformation that can be deployed each day to remove anything that has been created?


How can I export the Compute Optimizer list? How can I export the Compute Optimizer list?
compute

Hi,

I'm in the Compute Optimizer and simply want to get the data exported into a CSV/XLS file. When I click the Export button, I guess it'd be too easy to just let me download a file directly, so it says I need to put it in a bucket. I created a bucket, but apparently it doesn't have the right permissions so I can't put anything in it. Can anyone direct me to the proper way to make this work?

I found this guide, which I don't understand, but I did what it says and it seems to have worked for a single export. At least it says Queued instead of giving me an error, so maybe it's working...

https://docs.aws.amazon.com/compute-optimizer/latest/ug/create-s3-bucket-policy-for-compute-optimizer.html

However, if I try to export from the Compute Optimizer in the Global view, I get a screen that makes it look like I have to go create additional buckets for every region? And then go through the link above for every region? Is that right? Is there a way to make a bucket that can be accessed from everywhere or to export one big list from the Optimizer instead of making an export for each region?

Thanks.