So, our third year of engineering has just started, and we have a subject named Cloud and Distributed Systems.
Our professor asked a few questions about who knows about the cloud, what you all know, who has used it, etc. I was the only one who had used AWS EC2 and S3 instances, so the professor was impressed. After the lecture, she asked me if I would like to do a project on the cloud where we can use university machines to create a cloud environment for our class. The environment would have features such as storage, instances like EC2, etc.
She told me to learn about OpenStack software, how it functions, and the system requirements so we could ask the university.
Long story short, can you all suggest what I can do, how I can start, and if this project is scalable?
Also, a small P.S.: I am interested in making my career in cyber/cloud security. How can I integrate a security aspect into this project?
I've been doing research how cheap or expensive hosting an application on aws can be? I am a cs student working on an application currently with 14 prospects that will need it. To be drop some clues it is just collect a persons name,dob, and crime they have committed and have the users view it. Im not sure if a $100 will do without over engineering it.
I'm just curious about the technical side of it, do they have 16TB SSDs or do they use multiple ssds in raid or something like that?
I have a function that parses emails and send to my backend endpoint, while normal emails without attachments get parsed that ones with attachment does not even trigger lambda function ( Since there are no logs on cloudWatch )
When I receive emails I trigger an SNS and using that SNS notification my lambda parses the content in the email. I read somewhere that SNS can carry only 250KB data and therefore emails with attachments are not triggering my lambda function
I am not able to confirm this. And if this is true how should I handle emails with attachments?
I am currently going to 2nd year of college. I already know Python to a good level. So, should I learn boto3 or start Terraform? please guide me.
I have an internal company website which we made to only be accessible from certain IPs. We are planning to improve speed by optimizing its infrastructure. However, we were surprised to find out that previous guys put CloudFront in the back (as shown below).
Infra was first created in 2018/2019, I think. Was this a correct way in the past? Do you guys think there were any special reasons for this?
We are definitely thinking of putting CloudFront at the front, the bucket and ALB behind it, and limit access using WAF IP set rule.
Any insights would be appreciated. Thanks!
Hi,
I'm not trying to do anything specifically here, just curious to know about this versioning behavior.
If I suspend bucket versioning I can assume that for new objects version won't be recorded? Right?
For old objects, with some versions still stored, S3 will keep storing versions for objects with the same name when I upload a new "version"? Or it will override?
Trying to use elastic beanstalk for the first time. I terminated an environment and even deleted the application, but the environment is still visible, due to which I'm not able to create a new environment with that name. Its been 30 min.s already
Hi, I am new to AWS. I recently got a GPU instance: g4ad.2xlarge, and I am trying to install the AMD Pro driver for Windows but I am unable to do so. Has anyone been in my situation before, and how can I install the driver?
Hey Folks!
I've been using AWS Amplify for a while and there are a few things I absolutely love about it:
-
As a developer, I appreciate having my own infrastructure stack that I can easily delete if needed.
-
I love being able to edit both my frontend and backend simultaneously, with Lambda code updating remotely while staying available on the frontend.
However, there are also some things I really dislike, especially with Amplify's Genv2:
-
The forced use of GraphQL with their model, which has poorly documented and insufficient basic authorization principles.
I've been thinking about bypassing Amplify entirely and setting up my application stack using something like React + API Gateway + Lambda + DynamoDB. Ideally, I'd like to maintain a similar development approach where:
-
Every engineer has their own local environment.
-
Updates are reflected locally during testing.
-
It provides the same capabilities I enjoyed with Amplify.
So, I'm curious:
-
Is there a good template or AWS service that matches this use case?
-
Should I continue using Amplify but ignore its unnecessary features and instead define my own constructs to publish via CDK?
-
What alternatives have you guys tried and loved?
Looking forward to your insights and experiences!
Thanks!
HI. I am using a public Ipv4 address in my AWS VPC and it costing me $12 per month instead of 3.6 USD. What I am doing wrong?
Maybe someone can help with this. A department where I work can only generate data as a json file and they want it in our aws data lake. The json file has hundreds of nested tables. I'm looking to flatten it into many pyspark dataframes in a gluejob and then push the data to redshift tables.
I've only ever pulled csv or parquet files in using glue. Another problem is this department can't even give me the schemas ahead of time so I'm really hoping there's a way to get a crawler working here.
I'm also looking at documentdb as a solution instead. Would this be the path of least resistance? I have no experience with nosql databases. My thought here would be to use this to unpack the file and then use a glue job to push to redshift. Ultimately I'll need to join this data with our other tables to push to power bi for the business users.
Thanks for any insight I can get.
Hello there,
I'm planning to host a Retrieval-Augmented Generation (RAG) chatbot on AWS using the Mistral-7B-Instruct-v0.2-AWQ model. I’m looking for guidance on the following:
-
Steps: What are the key steps I need to follow to set this up?
-
Resources: Any articles, tutorials, or documentation that can help me through the process?
-
Videos: Are there any video tutorials that provide a walkthrough for deploying similar models on AWS?
I appreciate any tips or insights you can share. Thanks in advance for your help :)
Windows server on aws
I verified apache is running with Invoke-WebRequest -Uri $url -UseBasicParsing
It has a route to the internet; I can ping out but I can not ping the public ip address or load it from my browser
I am allowing enough ports in
Running into an issue where I need to perform a write transaction into a table but to prevent a race condition, I want to conditionally check if something in another table is not set to a certain value.
Problem is I can't really query this table before performing another API call to get the actual sort key.
Is there any way to do this?
What can I do to make this call be part of the same transaction?
Hi!
Not sure if this is the right place for this post but I figured it was at least a good place to start. I’ll try to keep the rambling to a minimum.
I WFH, my company uses AWS for all calls through the browser. I use a Bluetooth multipoint headset to listen to other things between these calls.
I can’t switch to my 2nd audio source if it’s detected that I’m still in a call on my PC in AWS. Perfectly fine, right? Nope. AWS continues controlling the system’s microphone for minutes past when the call ends.
This means that, for almost 5 minutes after the call has ended, I’m still waiting for AWS to release the microphone/stop using it so I can switch back to my 2nd device, & try to continue what else I was doing before another call rolls in & starts the process over again.
I need to somehow force the call to release(?), but nothing I've tried accomplishes that. I've refreshed the browser pages, completely closed them out & reopened them, even disabled/reenabled microphone access for desktop applications in the Bluetooth settings in Windows (AWS just takes control of the mic back as soon as permissions are reenabled 🙄)
There’s probably not anything I can do that will make a difference, but it’s driving me crazy so I had to ask lol
what a good aws iam policy tool that is for multi-cloud setup
EDIT: OK, I always forget about iam:PassRole
and (unrelated to this) permissions on service VPC endpoints, and end up frustrated with what's going on. I read the docs page multiple times and the note about iam:PassRole
right at the top just slid right off my smooth brain every time... Thanks to
I've got a (working) S3 bucket replication configuration between 2 AWS KMS (SSE-KMS) encrypted buckets with different keys (as they're in different regions, but the same account).
I've added to the IAM policy for the replication role permissions to s3:PutInventoryConfiguration
(at bucket level) and s3:InitiateReplication
(at object level, i.e. <bucket_arn>/*
), because my reading of the docs is that that is necessary in additional to the usual replication permissions for batch jobs to work.
I have a 'human' role/policy which I initially created with just the S3 'job' permissions (Including s3:CreateJob
), but in desperation at trying to get this damn thing working currently has s3:*
and kms:*
on resource *
. I have also, although I have no idea if it is necessary, granted sts:AssumeRole
to it on the replication role, and confirmed that I can assume the replication role through the AWS console.
Even with all that, when I go through the process of trying to create a batch job in the web console I get to the very end, click 'Create Job', and get the error
Access Denied. You or your AWS admin must update your IAM permissions. Learn more about [Identity and access management in Amazon S3](https://docs.aws.amazon.com/console/s3/accesscontrol)
That linked to page is useless. It's all about bucket and object access to S3 directly, there's nothing about batch jobs.
If I grant my human role *
on resource *
then I am able to create the job, which proceeds to work exactly as intended to replicate existing objects that were created before the replication policy was applied. I am aware of some other pitfalls I can come across with trying to write the [optional] manifest file or results of the batch job and the batch job role needing additional s3:PutObject
etc permissions on whatever bucket those go to, but I've already experienced those and they only come into effect after the batch job has been successfully 'created'. Generally the setup steps that must complete before you can 'run' it fail.
Given my human role already has s3:*
and kms:*
on *
, and I can successfully write objects to both buckets when assuming that role (proving that I have permissions to access both source and destination, in case that was a question), there must be some additional non-S3 and non-KMS permission needed to create batch jobs. But I cannot find this documented anywhere.
What the hell is the minimal permission set to just successfully create a batch job through the web console, regardless of whether the job initialises so it can be run or not...
Hi everyone,
I have a use case wherein I need to store thumbnail images in s3 and store the path in a database for those images. So the UI will have many such thumbnail images so I was wondering what is the best way to store these images as the image binary will be stored that will be rendered on the UI so I am assuming that will be many calls to s3.since it is a thumbnail image I was wondering if I should render it as an image url or store the image in s3?
What do you folks suggest?
Sorry my English is not that good,
This seems like such a dumb feature to not include, right? They support wildcards in the url, source IPs, but somehow not something as basic as the http method?
I think I can still accomplish what I need by using Cloudfront Functions and a KeyValueStore, but I'd at least like to understand the reasoning behind this...
Hello Guys, can Quicksight CUDOS dashboards work as a CloudHealth replacement or are there any missing features! I am also curious about cost for an AWS Org with 50 accounts and 500k monthly bill, does anyone has any approximate costs for such infrastructure? Thanks in advance
As there isn't that many free aws hands-on training available, we are in the process of setting up a training account and having users login to do stuff in that. My question is, what's the best way to reset the environment each day? Is there an easy way to just delete everything and have it start back from 0? Maybe some type of cloudformation that can be deployed each day to remove anything that has been created?
Hi,
I'm in the Compute Optimizer and simply want to get the data exported into a CSV/XLS file. When I click the Export button, I guess it'd be too easy to just let me download a file directly, so it says I need to put it in a bucket. I created a bucket, but apparently it doesn't have the right permissions so I can't put anything in it. Can anyone direct me to the proper way to make this work?
I found this guide, which I don't understand, but I did what it says and it seems to have worked for a single export. At least it says Queued instead of giving me an error, so maybe it's working...
However, if I try to export from the Compute Optimizer in the Global view, I get a screen that makes it look like I have to go create additional buckets for every region? And then go through the link above for every region? Is that right? Is there a way to make a bucket that can be accessed from everywhere or to export one big list from the Optimizer instead of making an export for each region?
Thanks.