What is Amazon Redshift?
Amazon Redshift is a cloud hosted database product from Amazon Web Services designed specifically for analytical workloads. It uses massively parallel processing and columnar storage to give tremendous processing power and can scale up to the petabyte storage range. It is very well suited to data warehousing applications of the type that might be done with technologies such as Netezza, Teradata or Vertica.
Why should I use it?
Redshift is lower cost than on site appliances, and is also more flexible because you can create and remove clusters when it suits your business. It is not necessary to commit to any payments beyond the charge for the next hour of use, making it ideal for short term high value projects where you can provision massive capacity and only pay for the hours you use it. There are also discounts for longer term commitments making long term continuous use very cost effective. In addition, it integrates very tightly with the wide range of products available in the broader AWS ecosystem, meaning it is efficient to set up capabilities securely and extend them where further value can be added to your business.
Can you help us work out how to use it?
Yes - our consultants have been working with Redshift since it was introduced, evaluating it for suitability to replace an existing enterprise data warehouse for a major communications provider. We have worked on all aspects of enterprise data warehousing with Redshift from architecture through operations optimisation, so we have extensive experience of the challenges and the solutions.
We can advise on everything from the details of the database structure and the control mechanisms and associated processes through to how it interacts with your existing infrastructure and choice of new tools to interact with it. Tools such as Tableau for data discovery and visualisation are typically very beneficial with Redshift and we can work with you to make the right choice for this.
Our founders are AWS Certified Solutions Architects and our associates have a rich depth of development and operations experience. We can draw on all of that when working with you to find out how we can add the most value to your business.
We can be engaged for shorter assignments to do an evaluation and/or initial architectural work, or to solve particular challenges you may have.
Can you help us implement it?
Yes - our consultants implemented a 30+TB scale Redshift enterprise data warehouse for a major communications provider, being involved at all stages from requirements gathering and setting the original architecture principles through design, development, test and operational optimisation, and including implementation of associated operations, development and reporting systems. There are several options for how we can be involved. For broad scope data warehouse projects we find we can add the most value by having one or more people working in an agile team alongside people who know your data and your business.
We recommend a very high level of automation to increase agility while remaining secure and stable. We have successfully been using automated deployment mechanisms for the code and the database which allow multiple deployments a day at the push of a button, all of which is fully audited.
We typically use CloudFormation and Puppet to control the configuration of environments - this allows us to actually take advantage of the cloud capacity to set up new environments at short notice without effort.
We also extensively use automated testing mechanisms (using DbFit/Fitnesse for business logic and python unit tests for infrastructure code) to allow rapid changes while remaining safe.
Can you help us operate it?
Yes - we have extensive experience in operating enterprise data warehouses, and can help setting up and optimising systems and processes and coaching in the use of these.
Can you help migrate from an existing data warehouse?
Yes - our consultants migrated a large enterprise data warehouse from SQL Server to Redshift, and we have developed a number of technical solutions and techniques that are particularly useful for migration projects. We can also contribute a lot from experience regarding project management and governance on these types of projects.
What extra value does Cloud BI Limited add?
We have especially deep experience in using Redshift successfully at large data scale and development complexity scale, and already have solutions to a lot of specific challenges that you may face. We want to share the benefits of a devops influenced approach to data warehousing. We’re easy to work with and want to see you succeed.
Download in PDF format: Amazon Redshift