So the functionality handled by lambda is to check if the requested file is processed or not from dynamoDB and send a response based on the status of the file being processed. Allocating the lambda function just to handle API gateway requests helps us satisfy the criteria to send a response within 29 seconds a request being made from API gateway or else it produces an error message, Adding any additional load on lambda can hinder the time constraints at hand. The first small process would be handling the API gateway request and revert back the response of the s3-presigned URL if the file present or else send file is processing. Since discussed above the computing and storage limitations of lambda, I tried to break the process into multiple small processes hence decoupling the application. What does my architecture now look like?.To overcome these limitations I have decided the following steps: The lambda function started running and given the appropriate output, but the heavy load of downloading a file and later converting it into mp3 has been pushing the limits of lambda, especially when given a large video file, making the implementation not much reliable and also due to limitation of the size of tmp repository to 512Mb handling a video larger than it lead to errors. To my surprise, the lambda function produced an error which is due to the fact that mp4 to mp3 conversion requires FFmpeg which is Linux kernel depended, unfortunately, we don't have access to the operating system of lambda so it was like a dead-end for conversion, but later I got to know about lambda layers and we could initialize a layer on top of the function with FFmpeg library, it was a great way to get introduced to lambda layers and I have implemented it. After going through the library and experimenting with python SDK I was comfortable working with the library, the interesting part is the SDK gives an option to download the directly best quality mp3 file makes its execution locally easy and not demanding much attention, problems araised when I tried to implement in the cloud.Īs my code started working locally I packaged my lambda function file and its dependent libraries into a zip file and uploaded it into lambda configured with python environment and tried to download the file into the temporary repository in lambda. The initial challenge while working on downloading content from youtube, is to find an open-source library, fortunately, I found a library called youtube-dl open-source library from GitHub. In this blog, I will be covering my experience while building the project like common pitfalls, design decisions I had to undergo while working on the project and maximum performance I could achieve given the constraint so let's dive into the project. Recently I had the opportunity to work on a music application, where I have developed a REST API to download the audio file given a video ID of the source youtube video.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |