Choosing the Optimal Data Storage Access Method for a Major Information Systems Developer
Analysts were tasked with describing methods for accessing data in S3 storage from the frontend through backend methods. During the exploration of various approaches, an alternative solution was found that did not require any additional programming code.
Customer
The client is a large developer of information systems.
Objectives
Business objectives of the project:
-
Cost Reduction: By finding an alternative solution that did not require programming, it was possible to avoid additional costs for development and maintenance of code.
-
-
Accelerated Implementation Time: The discovered solution allowed for a reduction in project implementation time, as there was no need to develop complex server logic.
-
-
Increased Team Efficiency: Processes of interaction between the frontend and S3 storage were optimized, simplifying data access and easing system support.
Task in the Context of Development:
-
Description of Data Access Methods: It was necessary to develop and document methods for secure and efficient access from the frontend to data stored in S3 through the backend.
-
-
Research and Selection of the Optimal Approach: Conduct an analysis of possible technical solutions to propose the best way to interact with S3 that met project requirements and minimized server resource usage.
Solution
We planned to describe methods of interaction between the frontend and S3 storage through the backend, ensuring secure data access while minimizing development complexity. The main task was to create an interaction architecture that could be easily integrated into the existing system.
Hypotheses:
Hypothesis 1: Develop a backend API to handle requests from the frontend and provide access to data in S3 through authorized methods.
Hypothesis 2: Use standard libraries and tools to minimize the need for creating complex server solutions.
Hypothesis 3: Explore the possibility of avoiding backend programming by utilizing built-in capabilities of S3 storage and access settings to manage requests directly from the frontend.
Why This Approach Was Chosen
The solution without programming was selected to reduce costs and accelerate implementation time. Utilizing standard S3 features, such as access policies and pre-configured rules, allowed us to avoid complex development and simplify future system maintenance. This approach enabled a focus on configuration and access security without the need to develop new server components.
Stages, Timelines, Tools, Numerical Indicators:
1) Requirements and Capabilities Analysis — 3 days:
– Define project needs.
– Research available tools for interacting with S3.
– Select potential approaches.
2) Implementation and Configuration of S3 Storage — 1 day:
– Set up necessary security policies.
– Configure access for frontend interaction.
– Integrate with the existing system.
3) Testing — 2 days.
Results
As a result of completing the task, the following goals were achieved:
1. Cost Reduction: Development and implementation of new backend components were avoided, leading to lower programming and subsequent system maintenance costs.
2. Accelerated Implementation: By utilizing built-in features of S3 storage, implementation time was reduced compared to the traditional approach of creating a backend API.
3. Security Optimization: The discovered solution provides the required level of security for accessing data in S3.
4. Simplified Support: The solution that did not require coding simplified system support and administration, as configuring access policies in S3 proved to be more flexible and quicker than setting up server logic.
5. Increased Team Efficiency: Simplifying data access processes allowed the team to focus on other critical tasks of the project, thereby enhancing overall productivity.
The outcome of the task was the successful implementation of a cost-effective and secure solution that fully meets business needs.
Technologies
- Nginx as a reverse proxy
- OpenStack Swift as S3 storage
Challenges
- Difficulty in Obtaining Information on Access Attempts from Frontend to Non-Existent Data: During the setup process, it became clear that tracking requests for non-existent objects in S3 required additional configuration and monitoring, as standard tools did not provide sufficient information about such attempts.
- Thorough Configuration of Data Access Rights: Ensuring secure access from the frontend required in-depth configuration of S3 policies and multiple checks to minimize vulnerabilities and protect data.