How To Go From A Project To A Successful Program?
The first couple of process automation projects is usually surrounded with excitement and enthusiasm. You have the necessary support from your sponsor and extraordinary help from the vendor as well. There is some leniency in the practices that you follow to deliver these pilot projects (a.k.a. PoC, quick win). Everyone has a vested interest at this point, everyone is trying to prove a point.
Center of Expertise (CoE)
Quite a few initiatives fail right at the beginning, and the remaining fail to grow beyond one or two projects. Over time, the level of enthusiasm will decrease, and you no longer will be able to use lightweight implementation processes. To stay on track and grow you will need to formalize your program. A Center of Expertise (CoE) can be an exceptional tool that helps you expand your program in a managed way.
Figure 1 - Digital Transformation Initiatives Failure Percentage
Unfortunately, the majority of the traditional methods of establishing a CoE, end up producing just one heavyweight body that slows down the progress while becoming a major cost center. A CoE should be a lean organization that works behind the scenes, without becoming a behemoth that simply consumes money.
There are three main components of a successful CoE.
- Strategy
- Delivery
- Infrastructure
We are not going into details of how to create a CoE, but the reason for adding this topic to the book was so that you understand what comes next. Figure 2 below shows major responsibilities of each component. Depending on the size of your program these responsibilities could be performed by three different teams, a single team or even an individual. Make sure to keep your CoE lean, and avoid making it a major cost center that can actually bring down your entire program.
Figure 2 - Center of Expertise Responsibilities
How to pick the right Process for Digitization?
Both value chain and customer journey maps discussed in the previous article will help you identify internal processes and improvement opportunities, but how do you decide which process automation opportunity to focus on first?
Automation Heat Map
An automation heat map can help you prioritize processes and activities that are good potentials for digital automation.
A great example of such an automation map can be found in McKinsey’s report “Where machines could replace humans—and where they can’t (yet)” [1]. The report uses an automation heat map to show the technical potential of automation in different sectors in the US.
Figure 1 shows a slightly modified version of the automation heat map for process automation purposes. It has two dimensions:
- Time Spent (%): This captures how much time is spent on a particular type of activity in a process and is represented by the size of the circle.
- Technical Feasibility (%): The second dimension captures technical feasibility of an activity or process being automated. This is represented by different colors.
Figure 1 – Process Automation Heat map
The table below provides a brief analysis of information presented in Figure 6.
| Process A | The majority of activities require someone to apply their expertise and working with unstructured data. Given current technologies the potential of automation is low. |
| Process B | Most of the work is predictable and a lot of time is spent on processing structured data so the potential for automation is high. |
| Process C | Even though the amount of time spent on predictable work and processing structured data is less in comparison to Process B, but the technical feasibility of automation is high. |
You can always add or change the criteria that are more suitable for your organization.
References
- Where machines could replace humans—and where they can’t (yet) - McKinsey Quarterly
Where To Begin Your Digital Process Automation Journey?
The first step on the path to digital transformation is becoming aware of your organization's current state. Knowing where you are and how things are in comparison to where you want to be and how things should be, is crucial for any planning. Unfortunately, most organizations do not have a good understanding of their current state, they do not have the necessary insights to identify processes that give them a competitive advantage or disadvantage.
Process Inventory
Organizations are made up of processes. A process inventory is a simple yet great tool that provides you with a snapshot of all those processes and activities in your organization. Infographic in Figure 1 summarizes key benefits of creating a process inventory.
You can use different tools to build a process inventory, but we are going to look at the two main ones. Both of these tools can be used independently or in conjunction with each other.
- Customer Journey Maps
- Value Chain
Figure 1 – Advantage of a Process Inventory
Customer Journey Maps
A customer journey map is a very simple idea: a diagram that illustrates the steps your customer(s) go through in engaging with your company, whether it be a product, an online experience, retail experience, or a service, or any combination. [1]
A customer journey map is a simple tool that allows you to visualize touch points between your customer(s) and your organization. It gives you an outside-in view of your organizational processes. This tool puts a customer front and center, which helps you create empathy for your customer. It provides you with insights into all the activities your customers perform, their motivations, questions, and barriers at each stage. In addition, not only does this tool identify all the organizational processes a customer interacts with, but it also captures their positive and negative experiences as they interact with those processes.
Customer journey maps can be end-to-end or focus on a single area. There is no one standard version of the diagram, normally it would be some sort of infographic with a timeline showing user’s experience. Figure 4 below shows a high-level customer journey map for a bank. [2]
Whichever format you choose, the goal, of course, is similar; learn about customer experience and find out what can be done to improve that. Just the act of creating a customer journey map helps in understanding the customer in a better way. Once you have created a customer journey map, identifying and prioritizing processes that can improve customer experience becomes much easier.
Figure 2 – Customer Journey Map for a Bank
Value Chain
Value Chain analysis was first discussed by Michael Porter in his 1985 book “Competitive Advantage”. Value Chain is a strategy tool that helps you visualize the process view of an organization [3]. It identifies all the primary and supporting processes involved in creating value for a customer. This gives you an inside-out view of your processes. The analysis reveals processes and activities where the competitive advantages and disadvantages of an organization are.
As part of the analysis, you look at the organization from the top down, create an inventory and models of all the processes, score them for the value they generate and identify all opportunities for automation. This exercise helps you understand how things are actually being done right now, what is generating value, what processes need improvements and of course what processes or activities can be automated. The value score can also help in prioritizing the work that needs to be done first.
Figure 3 - Value Chain Analysis
References
- Using Customer Journey Maps to Improve Customer Experience - Harvard Business Review
- The Retail Bank’s One-Stop Guide to Journey Mapping - Tara Litchfield
- Value Chain Analysis - Strategic Management Insight
Key Components Of A Digital Business Platform
Note: I am not associated with any digital business platform vendor. All information shared in this article is based on my experience and research.
Overview
Customer experience, operational process, and business model are the three key areas of digital transformation 1, 2. Regardless of which areas an organization focuses on, it needs a technology platform that will support its digital transformation goals. The term being used to define such platforms is Digital Business Platform (DBP) 3, 4.
Figure 1: Key Areas of Digital Transformation
According to Jim Sinur, DBP is a convergence of best of breed approaches 5. The list below shows key components that constitute a DBP 6.
- Process Management
- Decision Management
- Integrations
- Data & Analytics
- Internet of Things
- Artificial Intelligence
Figure 2: Key Components of a Digital Business Platform
All these components might not be available as part of a single unified platform. A single unified platform, of course, is preferred, but if your organization is heavily invested in some of these components already then you can always add-on to create a logical DBP of your own.
DBP vendors have taken different approaches as well to create their offerings. SoftwareAG, for example, had an integration platform, and over time acquired other products for their DBP offering 7. While Appian had a business process management software, they built additional features from scratch for their low-code application development platform 8, 9.
Now let’s take a deeper dive into each of these components.
Process Management
If you can't describe what you are doing as a process, you don't know what you are doing. - W. Edwards Deming
Processes are everywhere, they are what make organizations unique, they can make or break an organization. A process management component allows organizations to orchestrate the flow of work between people and systems. It provides tools for automation, execution, monitoring and optimization of processes.
All three key areas of digital transformation i.e. customer experience, operational process, and business model heavily rely on process management. A great looking website, mobile app or a digital product is required, they will (likely) result in great customer conversion rates, but in all cases, they will trigger or interact with some internal process. An organization will be able to acquire new customers of course, but customer retention will heavily depend on the quality of its internal processes 10.
Take for example the most common and relatively simple process of expense claims. It involves at least three sets of people and multiple systems. It can be done on paper or via email, but then you have no insights into the process. By automating this process, you can bring all the data into a single place, reduce data errors, provide visibility, and reduce cycle time.
Figure 3: Expense claims process
Decision Management
Management manages by making decisions and by seeing that those decisions are implemented. - Harold S. Geneen
Business users have traditionally been dependent on technology teams for implementing and maintaining business rules in applications. This dependency can be a cause of incorrect implementations and more importantly slow down any response to changes that are needed. A core component of DBP is decision management. Decision management provides a convenient way for both technology teams and business users to capture, automate, test and govern business rules 11.
Consider a simplified version of account opening process. Once an applicant submits new account application, the very first step is to check their credit score. Keeping this rule in the application will cause issues later on when changes are required. For example, organization policy changes, and now the new credit score requirement for opening an account is 725 instead of 700. If this rule was built into the application, then this update will require changes to the application. What if the same rule is used by multiple applications, then multiple applications will have to go through the same upgrade cycle.
On the other hand, if these rules reside outside of applications and are accessible to business users, then any change in organizational policy can be easily applied without involving the technology teams. Another benefit of keeping rules outside of applications is that you can have a single organization-wide rules repository that all applications can access, this ensures that organization policies take effect across applications without going through a long project of changing rules in each and every application.
Figure 4: Model of an account opening process
Integrations
A process cannot exist in a silo, in order to deliver real value, it has to integrate with other systems.
In most organizations, especially the ones that have legacy systems or have gone through mergers and acquisitions, there are a lot of systems that do not talk to each other. In absence of any system integrations, standard operating procedures are born that require people to do useless repeatable tasks. These tasks do not create any value for the organization but are necessary just to make things work. People end up filling gaps in technology instead of it being the other way around.
Integration component is a must-have for DBP. Two ways a DBP can reduce these useless human tasks and fill gaps in technology are by providing system integration and robotic process automation (RPA) capabilities.
- Integration Platforms: Integration platforms are specifically designed to handle different types of integrations. Not only do they support most protocols and have a large repository of pre-built connectors, they also have powerful mapping, transformation, and routing capabilities. All of these capabilities are necessary to build complex integrations. Another advantage of using an integration platform is that they prevent integrations from becoming point-to-point, so any changes in source system do not have a huge impact on client systems.
- Robotic Process Automation (RPA): RPA is not the traditional type of integration. RPA tools allow you to train software bots to mimic human actions e.g. open browser, log in, copy-paste data etc. So, in cases where integrations might take a while to build or are difficult to build e.g. legacy systems, RPA provides a great alternative 12.
Figure 5: Integrations platform architecture
Data & Analytics
If you cannot measure it, you cannot improve it. - Lord Kelvin
From acquiring new customers to winning elections, data has become the key. Answers to following and many other questions reside in the data.
- Where are different bottlenecks in the process?
- How are different rules performing?
- What are the common integration failure points?
- What is the most popular product?
- What other products did the customer look at?
- Can we recommend any other products to a customer?
- How many complaints have been received for a certain product?
Data and analytics capabilities of DBP should include features such as:
- Sourcing of data from different systems (ETL)
- Creating unified views of entities (DWH)
- Building reports, drill down, slice and dice (BI)
- Identifying key metrics (BI)
- Mining data to find patterns (Data Mining, Process Mining)
Figure 6: Data sourcing flow
Internet of Things
Analysts are forecasting that by the year 2020 there will be more than 20 billion connected things (devices) and the total revenue from the internet of things (IoT) will easily surpass $2 trillion 13.
A DBP needs to include capabilities of an IoT platform. An IoT platform allows developers to build, deploy and manage IoT applications from one central location in a secure manner. It provides necessary infrastructure and development tools so that developers focus on building applications and not waste time on setups 14.
To better understand how IoT can help organizations, consider the example of a Fleet Management company. Here are a few questions that a Fleet Management company might be interested in answering 15.
- Drivers: How are drivers performing? Are they making unnecessary stops? Are they driving over the speed limit and wasting fuel?
- Vehicles: How is a vehicle performing? Are there any issues that might need immediate attention e.g. low battery?
Without a connected fleet, all this data would come from drivers (which cannot be reliable), or by doing inspections only after a vehicle is back or something breaks down. Now, if the entire fleet was connected, then the vehicle would be transmitting this data automatically and using analytics all these questions and more could be easily answered.
Figure 7: Sample Fleet Management Dashboard
Artificial Intelligence
Artificial Intelligence (AI) is usually defined as the science of making computers do things that require intelligence when done by humans 16.
AI is going to have a huge impact on our future, but AI is not just a single technology, it is a combination of different research areas such as machine learning, deep learning, natural language processing etc. To better understand why AI capabilities need to be part of DBP and how it can help organizations, let's take a look at a couple of examples.
Medical Treatments: Daniel Kahneman a famous psychologist did a lot of work on the psychology of judgment and decision-making. In one of the experiments, his team worked with doctors to identify seven factors that they used to diagnose if an ulcer was malignant. Based on these factors, his team developed a model that could render diagnoses. Next, they fed both the doctors and the model same data, 96 different stomach ulcer images and asked for their diagnosis. In some cases, without telling the doctors they showed the same image twice at random to the same doctor. The resulting data was very interesting, not only did different doctors have different opinions but even the same doctor had different opinions for the same image. The model, on the other hand, had a better prediction rate. 17
Machine Learning (ML) does the same thing, it can consume huge volumes of data to create a model. If we look at medical treatments, the model can use all the data to recommend a course of treatment. We do not have to replace a doctor, but it can augment and help make better decisions.
Figure 8: Medical treatment recommendation
Customer Service: To keep existing customers happy and acquire new ones, a good customer service is extremely important for organizations. But depending on an organization's size, customer service department can be a huge cost center. Natural Language Processing (NLP) has improved a lot, now computer programs can not only understand standard sentences, but they can also interpret slang and tone used in the sentence.
As a result, the idea of chatbots is becoming extremely popular in the world of customer service 18. Chatbots can be trained to understand what people are asking and provide relevant responses. For cases where the question is outside of a chatbots training, it can simply redirect them to a specialized human representative. Chatbots can augment customer service representatives, and reduce cost by taking care of the standard queries.
Figure 9: Chatbot sample interaction
Conclusion
It’s so much easier to build code with objects, like Lego blocks, than it is with lines of code. - Matt Calkins
We looked at six key components of a DBP, but before I close this post, I would like to take a quick look at different characteristics of a DBP.
- Unified
- Low-Code
- Cloud-based
Characteristics listed above result in agility, smaller learning curve, reduced support and maintenance costs, reduced time to market and simpler licensing plans.
Organizations need to have Digital Business Platform in their toolbox if they not only want to survive but thrive in this fast-paced and disruptive digital world.
References
- The Nine Elements of Digital Transformation by George Westerman, Didier Bonnet and Andrew McAfee (MIT Sloan Management Review)
- What is Digital Transformation by Adeel Javed
- Is the Digital Business Platform the Future of BPMS?
- Is the Digital Business Platform What BPM Has Been Aiming for All Along?
- Creating a Digital Business Platform by Jim Sinur
- bpmNEXT Keynotes by Keith Swenson
- SoftwareAG Digital Business Platform
- Appian Low-Code Application Development Platform
- Why Use Low-Code Application Development Platforms? by Adeel Javed
- Why Digital Transformation Can't Proceed Without Operational Process Transformation, by Adeel Javed
- A Realistic View of Business Rules Engines
- How To Make BPMS And RPA Work Together
- Gartner Says 8.4 Billion Connected "Things" Will Be in Use in 2017, Up 31 Percent From 2016
- Building Arduino Projects for the Internet of Things: Experiments with Real-World Applications, by Adeel Javed
- 5 Ways The Internet Of Things Revolutionizes Fleet Management, by Jay Krishnan
- What is Artificial Intelligence?, by Jack Copeland
- The Undoing Project, by Michael Lewis
- Ultimate Guide to Leveraging NLP & Machine Learning for your Chatbot, by Stefan Kojouharov
How To Sell BPM In An Organization
BPMTips.com recently did a great post about selling BPM in an organization. They asked more than 20 BPM experts this question, "How to sell BPM in an organization?".
I found following common themes in most responses.
BPM is hard to sell
Executives are not concerned about tools and methods, they are interested in outcomes and value
Less ‘selling’, more ‘benefits’
Have an executive sponsor who is committed to BPM over the long term.
Word Cloud generated from all the responses also provides nice insights.
Read the complete article on BPMTips.com to gain valuable tips.
How To Monitor Multi-System Business Processes
Organizations use processes to manage their business. How each process gets implemented falls into four main categories.
- Paper-based
- Email + Spreadsheet
- Multiple Systems
- BPM Software
Organizations have invested a lot of time and money in processes that fall into category 3. Bringing all those processes into a BPMS is not an easy or quick job. I have seen a single process that goes through 20+ systems from start to end.
Let's take an example of order fulfillment, a fairly common process. The diagram below shows lifecycle of an order as it moves through various systems. Most of the times you will find core business of an organization distributed into multiple systems in a similar manner. Work, of course, gets done, but at any given time you cannot find exactly what is the current status of the process. And to generate even simple reports, data first needs to be consolidated from all the systems manually or through some sort of batch job.
So, how do you monitor a process that resides in multiple systems and ensures that it is running optimally considering it does not provide a single view. You build a shadow process.
The idea of shadow process is simple; you design the end to end process with all major milestones without actually implementing all the functionality in a BPMS.
Implementation Steps
Here are high-level steps for implementing a monitoring system using shadow processes.
- Define process with major milestones
- Level of milestone granularity is directly proportional to effort required
- Use milestones that make sense for reporting
- Find a common id that will allow you to uniquely identify an instance of the process in all systems
- Build services that either receives events from external systems or fetch data from external systems
- Build services that advance the process instance based on event data
- Log key event data e.g. instance id, business data, start time and end time etc. These can then be used by any reporting tool to generate meaningful insights
Compared to implementing all the processes in a BPMS, this type of monitoring solutions is much quicker to implement, provide end to end process view and helpful insights for future optimization.
The Case For Agile Methodology
Almost everyone (in the US) is familiar with the Healthcare.gov website and the drama that surrounded its launch. Interestingly enough the front-end was developed in 3 months using the Agile methodology, but back-end that had to connect multiple systems to IRS and Insurance companies used Waterfall methodology, the problem? Well, everyone decided to test all these systems close to the end, because that’s when you test them, at the end when there was not enough time, so they cut short testing and launch the website. As one can only imagine, the back-end systems simply did not work.
People who fixed that mess used Agile.
Rigid Plans vs. Incremental
Planning is important for combat, but the moment first bullet is fired, your plan goes up in smoke
Source: President Eisenhower
Waterfall relies on such “color-coded” and “detailed” plans that are good to look at but lack reality.
The idea that 100% of project requirements need to be captured and signed-off before analysis stage can be completed and design started, and that there will be no changes until the project is deployed is simply flawed.
Agile on the other hand is a time boxed, iterative approach that simply says instead of doing everything at once, build your software in 2-4 week iterations. Small bangs instead of a big bang. You still go through all stages that a waterfall project goes through, but in smaller chunks, so there is no compromise on design, quality or documentation.
Everything vs. High Priority
Waterfall processes require to implement 100% of requirements otherwise it is not accepted, so essentially all requirements are equal.
He who defends everything defends nothing.
Source: Frederick the Great
Time-boxed nature of Agile forces customers to prioritize, which means the most important stuff gets done first, and the least important is saved for last.
Delayed Feedback vs. Early Feedback
I worked on a project for a client where the waterfall was part of their DNA. After 6 months of analysis we took requirements to stakeholders for sign-off, and the very first comment, due to recent audit findings we need to add new requirements in scope.
Due to recent audit findings we need to add new requirements in scope
Back to the drawing board, after another 4 months we request sign-off, and once again due to org changes, we need to redo the whole analysis phase.
Let’s assume you successfully complete the development, and during the testing phase, your users interact with the software for the first time in months or even years. Usual customer response, you have developed what was in the requirements document but well this is not what and how we had imagined the system.
Not just users, this is the first time development team gets to test their architecture and design.
So Agile recommends to test and get feedback from users earlier and regularly, it welcomes the change.
Wasted Effort vs. Continuously Improve
Another issue that stems from waterfall methodology is that the expectation is 100% of requirements need to be implemented before software can be deployed. Study of hundreds of software systems show that users only use 20 % of the features, yet we deliver 100% of features, which means that the effort both money and cost spent on that 80 % of features might not have been necessary.
How do the iterative nature and requirements prioritization help? Agile recommends shipping early when you have completed 20% of the features that will deliver 80% of the value then deploy the software in production. And based on real feedback, re-prioritize remaining requirements, and then deliver 20% of those features. Within 2-3 releases you would have delivered more than 150% of the value. And you may or may not even need to deliver rest of the requirements and focus the effort on something else.
You only need to deliver what is needed, e.g. during the Iraq War, U.S. spent huge sums of money to build a chicken processing plant that did not get used, so finally, they asked villagers what they wanted? Villagers wanted a simple, foot bridge so that they don’t have to go all the way around to get supplies. It might not be as impressive but that was exactly what added value and was the need.
Poor Visibility vs. Improved Visibility
In waterfall being half way through the project does not necessarily mean you have implemented half the feature, but with Agile you always know how much work has been completed and how much is left.
You are also going to hear a lot of negative feedback about Agile, in my opinion, it is mostly propagated by people who do not do Agile the right way.
How BPM Cycle Has Evolved
Emiel Kelly in his post "Customers don’t care about BPM cycles" provides insights into how the age old BPM cycle has evolved over the years. Almost everyone in the BPM industry is familiar with the following cycle, this is usually the first thing we learn and then teach.
Source: Emiel Kelly
Following cycle shows the more mature and evolved cycle of BPM. Interesting points to note:
- Having both Design and Model steps in the cycle has always been confusing, so this definitely removes that confusion.
- Execute step is more defined now and gives an idea about what really happens.
- Removing Optimize makes sense because that is the whole goal of the cycle and is not just an activity.
- Of course adding the swim lanes clarifies who does what.
Source: Emiel Kelly
Getting started with Xively®: Data Publish
This post provides an overview and learning objectives of Chapter 12 from my book Building Arduino Projects for the Internet of Things.
Introduction
IoT platforms provide developers with the ability to develop, deploy and manage their IoT applications from one central location in a secure manner. IoT platforms expedite the development process by providing required tools in a cloud-based environment, which means developers do not spend time on setups. A good IoT platform would ideally include most of the tools that we have covered in previous 11 chapters such as MQTT broker, HTTP server, REST API support, database to store sensor data, Node-RED for complex orchestrations, device location, secure communications, reporting, analytics and easy to use tools for building the web and mobile apps etc.
In this chapter, you are going to cover a popular IoT platform Xively. You are going to build a Soil Moisture Control system that sends out an email alert whenever the moisture level of soil falls below a certain threshold. The figure below shows a high-level diagram of all components involved in this system. The first component is an Arduino device that monitors soil moisture level and publishes a message to Xively. Second and third components reside on Xively platform. With some basic configurations, the platform will be able to receive, store and display data sent by the sensor.
Learning Objectives
At the end of this chapter you will be able to:
- Read soil moisture sensor data
- Setup Xively to receive sensor data
- Setup trigger in Xively to send an email using a Zapier task
- Write code to read sensor data and publish it to Xively
Final Product
IoT Patterns: Machine to Machine - Energy Conservation System
This post provides an overview and learning objectives of Chapter 11 from my book Building Arduino Projects for the Internet of Things.
Introduction
As IoT technology evolves, and machines become smarter and capable, the need for human intervention will reduce. Machines will be able to autonomously respond to alerts generated by other machines.
In this chapter, you are going to build an Energy Conservation System that will show how two machines can communicate. The figure below shows a high-level diagram of all components involved in this system. The first component is an Arduino device that monitors light brightness levels and sends an alert whenever the levels are low. The second component is an MQTT broker that helps avoid point-to-point communication. Multiple devices can communicate with each other without knowing each other's identities. The final component is another Arduino device that controls lights, if light sensor publishes a message to MQTT broker that light brightness level is low then this device will automatically turn the lights on. In real life, you can see this system being utilized to turn street lights on or off only when they are required instead of doing that at scheduled times.
Learning Objectives
At the end of this chapter you will be able to:
- Read data from a light sensor
- Publish message to MQTT broker
- Control LED lights
- Subscribe Arduino to an MQTT broker