You can store your files and data in AWS S3. That is what is usually known by everyone about AWS S3. Well, S3 is known to be one of the older services by Amazon since the days of game changing Alexa Skills and revolutionary Lambda functions. Here in S3 you can easily store all type of files ranging from doc to pdf and files of all sizes ranging from 0B to 5TB.
In simpler words, AWS S3 is an object oriented storage system where you store and save can all kinds of files as object and not file. Many big tech wigs across the globe uses S3 and one of it is dropbox which is known to save metadata of files in own service but they still save their main data in S3. The reason behind preference of S3 is that it is not at all expensive and is 99.9% available. Additionally, it also let you change services which saves data and can charge almost $0.01 per GB.
In this article, we will be talking about how to use S3 in nodeJS application and have detailed steps on how to build NodeJS based application which can write any file to AWS S3.
Well, to make you aware it is to be known that AWS has official package which can expose S3 apps for NodeJS apps, making it quite easier for developers to access S3 from their apps.
STEPS TO BUILD NODEJS BASED APP
- Set up node app
A node app usually has two files, package.json (for dependencies) and a starter file (like app.js, index.js, server.js). Follow below commands:
mkdir s3-contacts-upload-demo
cd s3-contacts-upload-demo
touch index.js .gitignore
npm init
After this, you’ll get a folder named s3-contacts-upload-demo which has three files in it, package.json, .gitignore andindex.json. Use this file to add file in your .gitignore file, to avoid get committed to github or some other version control.
- Install dependencies
Install the NPM package by following command: npm install –save aws-sdk
Upon its successful installation, you can check your package.json file, as have aws-sdk listed in “dependencies” field. The installed npm package is useful in accessing any AWS service from nodeJS app, one being S3.
- Import packages
Once npm package is installed, import it in your code by following below commands:
const fs = require(‘fs’);
const AWS = require(‘aws-sdk’);
const s3 = new AWS.S3({accessKeyId: process.env.AWS_ACCESS_KEY,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY});
Now, fs package has been installed which is used in writing file data in the app so you can have your S3 instance which can access all the buckets in your AWS account.
- Pass bucket information and write business logic
Bucket information and writing business logic entails how to upload file to S3 Node JS in which keyis for subfolder. In simple words, if bucket name is “test-bucket” and one want to save file in “test-bucket/folder/subfolder/file.csv”, then value of Key should be “older/subfolder/file.csv”.
- File to upload to S3
Create a file, for example: contacts.csv with some data written in it. Read the contacts.csv file, using fs module and save it to S3. You can now run this app by following below command:
AWS_ACCESS_KEY=<your_access_key>
AWS_SECRET_ACCESS_KEY=<your_secret_key> node index.js
We have passed our AWS keys as environment variables.
If there is no error in aforesaid snipped, bucked should have the file and now you can upload a file to AWS S3. You can find your complete project here.
FINAL THOUGHTS
You can a lot of things with the package like listing out of objects and buckets, set permissions on bucket, or create, get or delete the bucket, and much more. It is strongly recommended that one should go through the doc for APIs as they’re well explained with parameters so you can easily pass to each API and response format they return.
