Aggregating data for graphs, analysis, portfolios, or even machine learning can be an arduous task and difficult to scale. In this article, I will go over MongoDB’s new(ish) $merge pipeline that I feel resolves a lot of these scaling issues and automates certain design practices that previously took a lot of custom development to accomplish, however, Mongo’s documentation fails to provide extrapolated examples or multiple use cases. This article will be diving heavily into MongoDB’s aggregation operation. …
Recently I’ve had to redesign an async queuing system. After researching several options, I went with AWS Simple Queue Service(SQS) and Simple Notification Service(SNS). With a combination of these two services, I was able to duplicate and improve the previous (and failing) RabbitMQ set-up. The great thing about the AWS offering is that they are built from the ground up for micro-services. They abstract the vast majority of the complexity of a complicated async messaging system, from handling horizontal scaling via the message waiting period to Dead Letter Queues and a multitude of other features, right out of the box.
As per AWS suggestion on scaling, they recommend scaling horizontally across multiple services as well as within a single service by utilizing multiple threads. …
GraphQL offers a very compelling solution to a common microservice problem. By implementing an API-Gateway using GraphQL, we can significantly improve the transactions between the consumer and API as well as ease the growing pains as the API and consumer requirements evolve over time.
However implementing a GraphQL gateway presents another problem. GraphQL’s query and mutation implementation are both very interesting and very powerful, but seem to be two separate ideas combined under one specification with very different requirements. By splitting them into 2 separate services. We see improvements to ease of use, documentation, manageable code, scaling, resiliency and authorization.
Microservice architecture solves many problems while introducing a few. …
It seems like such a simple request at first, “I want data to be ordered by rank or priority, and I want users to be able to re-order the data based on which is most important”. This is an extremely common scenario in app development. You will find this problem everywhere from Todo Lists and task management systems, to complex and specific use-cases. The solution might seem simple at first, depending on your language/db solution, there might even be native support for a priority queue, or perhaps an array could function here. …
In this blog post I will go over how to interact with S3 objects via the AWS-CLI. In this case I will follow methods I took to move objects into a different organizational structure to support a Cloudfront distribution standard.
The first step to dealing with the aws-cli is to set up and verify that you have the security credentials to handle any operations you plan on using. To do this, go to the IAM Management Console and head over to Users. …
I was recently quote mined by a strongly worded opinion piece on Bloomberg. I felt a degree of guilt for being a part of a message that pushed people away from a potentially life changing opportunity. The piece, I won’t link it here but you can easily find it with a search, provided a scathing review of certain software engineering bootcamps, but it fell short from painting a clear picture of what the ecosystem looks like.
To be fair, the sentiment of the article is true, many bootcamps are little more than scams and bootcamps in general aren’t for everyone. However, the good bootcamps set themselves apart and present a new paradigm in education; one that allows people to reinvent themselves in a world that is increasingly automated. This can help people that don’t normally have access to universities, are too old, or even those who have graduated but lack applied experience necessary to get a job. …
Uploading files and varying types of media can always be a headache. I was recently tasked with building out a media service; a dedicated Go service that would handle uploads to an s3 bucket. The documentation isn’t completely clear and most of the examples and tutorials are outdated, so I am posting a small snippet of basic working code using the AWS SDK
The first step to setting up the AWS-SDK is to define your credentials. Eventually you’ll want to do this using an environment variable or the /.aws/credentials file. For now we are going to use static credentials in code Do Not Upload To GitHub or use in Production or Development. …
This is the third and final installment of my blog trilogy on ES6 based on the information presented by Aaron Frost on Pluralsight. If you haven’t seen the talk, check it out! In this episode, we are going to take a glance at promises, generators, and modules!
Promises are one of the most exciting features being presented in ECMAscript6. If anyone has ever worked heavily in asynchronous code, then you are probably very familiar with Callback Hell, let me show you what it looks like:
Callback Hell commonly occurs during chained asynchronous functionality. Without setting the callback properly, the next step would get called before the previous step completed, usually causing an error or crashing the program. …
For the second edition of my ES6 blog trilogy, we are going to take a quick peek at some really cool features such as, symbols, maps, collections, default parameters, and classes! Again, if you haven’t seen it yet, check out Aaron Frost’s ES6 talk on Pluralsight.
Classes provide a new syntactical way to create classes and subclasses that is both intuitive and easy to follow. It is important to note that the new class feature offers no new functionality or performance gains (as of yet). Everything you can do with classes you can also do with the current pseudoclassical instantiation pattern and prototype chaining. …
Wow! I am so excited for ECMAscript 6 I can barely contain myself. Only one month until it is officially released. I just finished watching Aaron Frost’s overview of ECMAscript 6 over at Pluralsight. If you haven’t tuned in yet, check it out here. Over the next three blog posts, I am going to quickly summarize some of the most exciting features of ES6.