Movatterモバイル変換

HOW TO DESIGN A BACKENDFOR THE IOTİBRAHİM GÜRSES

WHO AM I?▸ Graduated From Bilkent University in 2011▸ Vakıfbank, Gate Elektronik, T2 Yazılım, OpsGenie,Hazelcast, Arçelik.▸ Currently working on IoT infrastructure @Arçelik▸ Co-Founder of Ankara Cloud Meetup

SERVERLESS COMPUTING WITH AWSHTTPS://WWW.YOUTUBE.COM/WATCH?V=LEPQXTOWDJS

DEVOPS CULTURE AND PRACTICESHTTPS://WWW.YOUTUBE.COM/WATCH?V=D3E0XJCRWCE

AGENDA▸ IOT▸ DevOps▸ 12 Factor App▸ Cloud▸ Microservices▸ Q&A & Demo

FROM MOORE’S LAW TO METCALFE’S LAWMetcalfe's law states that the value of a telecommunications networkis proportional to the square of the number of connected users of thesystem

WHAT IS IOT?▸ The network of physical object that contain embeddedtechnology to communicate and interact with their internalstates or the external environment. (Gartner)▸ The term is coined by Kevin Ashton in 1999 inProcter&Gamble▸ Also called M2M, Industrial Internet, Web of Things,Internet of Everything, Industry 4.0

A TYPICAL IOT DATA PROCESSING ARCHITECTURESource : Internet of Things: Principles and Paradigms, Elsevier Science, 2016

PROCESSING DATA FROM THE EDGE▸ Collect▸ Instrument apps▸ Deliver events to analytics service▸ Receive and store many live data streams▸ Analyze▸ Real-time and historical analysis of event streams▸ Aggregations, pivots and patterns▸ Consume▸ Publish analytics in a consumable format▸ Inform and inﬂuence▸ Make better decisions

DEVELOPMENT BEFORE DEVOPS▸ DevOps is a new term that primarilyfocuses on improved collaboration,communication, and integration betweensoftware developers and IT operations. It’san umbrella term that some describe as aphilosophy, cultural change, and paradigmshift. Figure shows developer throwingcode "over the wall" Historically manyorganisations have been verticallystructured with poor integration amongdevelopment, infrastructure, security andsupport teams. Frequently the groupsreport into different organisationalstructures with different corporate goalsand philosophies.

WHAT DEVOPS BRINGS▸ Today, these old divisions are breaking down, with the IT anddeveloper roles merging and following a series of systematicprinciples:▸ Infrastructure as code▸ Continuous deployment▸ Automation▸ Monitoring▸ Security

INFRASTRUCTURE AS CODE▸ Repeatability (Humans make mistakes)▸ Agility (Roll forward or roll back easily)▸ Auditing and Security (Paper trail and permissions)

AUTOMATION AND CONFIGURATION MANAGEMENT▸ Provisioning (CloudFormation, OpsWorks, BeansTalk)▸ Declarative (Loosely coupled to implementation)▸ Conﬁguration (Chef, Pupper, SaltStack, Ansible, DSC)

MONITORING AND SECURITY▸ Processing all systems logs in real time.▸ Logs should be considered as events▸ Security can inject analysis tools to dev pipeline.▸ Testing is not optional in devops.

DEVOPS▸ Do not write code and toss it to ops and testing team▸ Do not repeat task manually▸ Rise of devops tools(Chef, Puppet, Ansible)▸ Spend time developing business code instead ofinfrastructure code (NoOps)

MOVING LEGACY APPS ON CLOUD▸ Asset Hosting▸ How do you deal with uploaded content? (images/videos/music)?▸ Session Management▸ How do you deal with session data? Session replicationwill be a necessity, sticky session is bad for scalabilityand availability

MOVING LEGACY APPS ON CLOUD CONTD▸ SQL▸ What considerations are there SQL? (How to handlestored procedures)▸ NoSQL▸ How can you take advantage modern trends of NoSQL?

MOVING LEGACY APPS ON CLOUD CONTD▸ Caching▸ How do you incorporate modern caching techniques?▸ Async Processing▸ How do you handle long running processes?

WHAT IS 12 FACTOR APP?▸ It is a methodology for building SaaS application▸ Tries to deﬁne systematic problems in app development▸ Tries to deﬁne a set of conceptual solutions to thoseproblems

GENERAL PROPERTIES OF 12 FACTOR APP▸ Uses declarative format for setup automation.(Easyorientation for new joining devs)▸ Has a clean contract with underlying operations system(Increases portability)▸ Is suitable for deployment on modern cloud systems(CloudNative app, also no need for an army of ops guys todeploy and maintain the app)

12 FACTOR APP▸ Code is version Controlled▸ Always tracked in version control system▸ 1:1 relationship between code base and app▸ Many deploys of given app▸ Codebase same across deploys version may differ

12 FACTOR APP▸ Dependencies are declared and Isolated▸ Never assume system-wide packages▸ Dependency declaration manifest▸ Isolated so no dependency leak from system▸ Helps new developers

WHAT WE DO?We use maven. A newdeveloper can start workingby simply typing singlecommand `mvn clean install`and all library dependencieswill be installed.

12 FACTOR APP▸ Conﬁguration is Stored in the Environment▸ Should store in env variables▸ Should not be constants in code▸ Ideally not in conf ﬁles▸ Avoid grouping as environments

WHAT WE DO?▸ All environment variable and configuration information isstored over AWS and all applications including mobileclient and wifi-card gets their configuration informationfrom a single place.

12 FACTOR APP▸ Backing Services as Attached Resource▸ Services consumed over the network▸ No distinction between local or third party services▸ Keep Dependencies de-coupled▸ Attach and detach at will

WHAT WE DO?▸ We use AWS services for both SQLand NoSQL data storage(RDS,DynamoDB)

12 FACTOR APP▸ Build and Run Stages are separated▸ Impossible to change code at runtime▸ Releases should have IDs▸ Build may be complex, started by Devs▸ Run is simple and completely unattended

12 FACTOR APP▸ Application Executed as Stateless Processes▸ Share Nothing (Universal Scalability Law)▸ Persisted data in stateful backing store▸ Memory and File System is for cache only▸ Avoid sticky Sessions

WHAT WE DO?▸ We implemented stateless serverless architecture withAWS API Gateway and Lambda.▸ Each request to cloud is executed within a Lambdafunction inside a isolated stateless container

12 FACTOR APP▸ Services Exported via Port Binding▸ Self Contained▸ Embedded servers▸ Listen on speciﬁc port▸ Very speciﬁc and idealistic

12 FACTOR APP▸ Application scaled out via process model▸ Processes are ﬁrst class citizens▸ Work assigned to process type▸ Applications have process that span servers▸ Use OS process managers not deamons

12 FACTOR APP▸ Processes are disposable▸ Can be started or stopped at any time▸ Minimal start up time, graceful shutdown▸ Worker processes return to work queue▸ Robust against sudden death

12 FACTOR APP▸ Parity Between Application Environments▸ Avoid time/personnel/tool gaps▸ Design for continuous deployment▸ Very important for backing services▸ Containers and conﬁg mgmt. makes this easier.

12 FACTOR APP▸ Logs are stream of time-ordered events▸ App is never concerned with storing log ﬁles▸ Execution environment capture logs▸ May be routed to ﬁle, watched, sent to external service

WHAT WE DO?▸ We use AWS CloudWatch to monitor system logs.

12 FACTOR APP▸ Management Task Run as One-off Process▸ Run in identical environment▸ Separate out as scripts that are source controlled▸ Don’t run from local terminal▸ Don’t run directly against the database

ADDITIONAL DEVOPS DESIGN CONSIDERATIONS▸ Rely on sync messaging▸ Compose applications out of service▸ Assess portability requirements▸ Embrace the abstractions

DEVOPS ANTI-PATTERNS▸ Relying on the local ﬁle system▸ Building services that scale up▸ Trying to change code server side▸ Manually coordinating builds▸ Hard-coding conﬁguration▸ Cramming everything into one app

DEVOPS CONCEPTS BEFORE FAILURE▸ Chaos Monkey▸ Blue/Green - Canary Deployment▸ Dependency Injection▸ Andon Cords▸ The Cloud▸ Embedded Teams

DEVOPS CONCEPTS AFTER FAILURE▸ Blameless Postmortems▸ Public Status Page▸ Developers on Call▸ Incident Command System

CAMS MODEL▸ Culture▸ Automation▸ Measurement▸ Sharing

KAIZEN’S GUIDES▸ Good processes bring good results▸ Go see for yourself (gemba)▸ Speak with data, manage by facts▸ Take action to contain and correct root causes▸ Work as a team▸ Kaizen is everybody’s business

LEVELS OF MATURITY OF DEVOPS PROCESS

CLOUD APPLICATION DELIVERY MODELS▸ IaaS (Infrastructure as a Service) - Host▸ PaaS (Platform as a Service) - Build▸ SaaS (Software as a Service) - Consume

AWS IOT COMPONENTS▸ Device Gateway▸ Enables devices to securely and efﬁciently communicate withAWS IoT.▸ Message Broker▸ Provides a secure mechanism for things and AWS IoTapplications to publish and receive messages from eachother. You can use either the MQTT protocol directly or MQTTover WebSocket to publish and subscribe. You can use theHTTP REST interface to publish.

AWS IOT COMPONENTS▸ Rule Engine▸ Provides message processing and integration with other AWS services.You can use a SQL-based language to select data from messagepayloads, process and send the data to other services, such as AmazonS3, Amazon DynamoDB, and AWS Lambda. You can also use themessage broker to republish messages to other subscribers▸ Security and Identity Service▸ Provides shared responsibility for security in the AWS cloud. Your thingsmust keep their credentials safe in order to securely send data to themessage broker. The message broker and rules engine use AWS securityfeatures to send data securely to devices or other AWS services.

AWS IOT COMPONENTS▸ Thing registry▸ Organizes the resources associated with each thing. You register yourthings and associate up to three custom attributes with each thing. Youcan also associate certiﬁcates and MQTT client IDs with each thing toimprove your ability to manage and troubleshoot your things.Securityand Identity Service.▸ Thing Shadow Service▸ Provides persistent representations of your things in the AWS cloud. Youcan publish updated state information to a thing shadow, and your thingcan synchronize its state when it connects. Your things can also publishtheir current state to a thing shadow for use by applications or devices.

MOBILE SAMPLE BACKEND SERVERLESS ARCITECTURE

MICROSERVICE▸ Is there a formal deﬁnition for microservice architecture ?▸ No▸ What is the Difference between monolithic and microservice styles?▸ Easy to maintain▸ Deployment▸ Scaling

ADVANTAGES▸ Can use right tool for the job▸ Can replace entire components easier▸ Can scale speciﬁc components▸ Super cloud friendly▸ Will push you DevOps

CHALLENGES▸ Distributed/versioned conﬁguration▸ Auto conﬁgurations and refresh on runtime▸ New services can auto register at startup▸ Service registration and discovery▸ Centralised log management▸ Collects and visualise log events from distributed processes▸ Circuit Breaker (Bulk Heading)▸ Prevent problems with chain of failures▸ Security

SUN’S FALLACIES OF DISTRIBUTED COMPUTING▸ The network is reliable.▸ Latency is zero.▸ Bandwidth is inﬁnite.▸ The network is secure.▸ Topology doesn't change.▸ There is one administrator.▸ Transport cost is zero.▸ The network is homogeneous.

ANY ORGANIZATION THAT DESIGNS ASYSTEM WILL PRODUCE A DESIGN WHOSESTRUCTURE IS A COPY OF THEORGANIZATION’S COMMUNICATIONSTRUCTURE.Melvin ConwayCONWAY’S LAW

Movatterモバイル変換

Change Language

How to Design a Backend for IoT

Embed presentation

Recommended

More Related Content

What's hot

Similar to How to Design a Backend for IoT

Recently uploaded

How to Design a Backend for IoT