Reengineering Customer Experiences with Outage Analytics

Intro how's everybody doing I am your last meeting before you can go home so hey everybody I'm John vershinski I'm from DTE uh thanks for attending the session this afternoon um here to talk about analytics using improved customer communication so uh big words uh could be anything across the board for DT but this is very much specific to a lot of the customer experience related to outages so uh we'll kind of get to some of the details on what I mean by that um just a little bit about uh um like I said we're going to talk about power outages we're going to talk about a project that we've been leading over the last couple years called EFC error-free Communications and then how we utilize analytics for outage management um you know not just from uh analytics from a couple perspectives one data quality and understanding what our what our experiences with our customers and the types of information that is related to that um and then just wanted a little bit of knowledge in our partners and the tools and systems and people we had in uh to use to make sure this project was successful um a little bit about DT guys all know we're from Detroit um you know we have two sides electric and gas 2.3 million electric customers um you know 1.3 million uh gas customers mostly across Southeast Michigan our gas size across the entire State but anyways um and then you know sprinkle here or there um and then uh just From outer's perspective so we talk a lot about you know this this initiative or what I'm talking about today is specifically around outages so we want to give you a little bit of insight a type of outages problems that customers have across the industry not just DT but little did you know I you guys all know 62 percent of our problems are related to weather but not just the weather it's also related to trees damaged winds and tree trimming so a lot of the things kind of fall together trees break lose your power so a lot of the things that you know we see from our side day to day is you know during storms and and one of the important things that we do at DTE is you know do a lot of tree trimming so you know I'm on the I.T site I don't work in the field so you know be nice to me a little bit I'm just a good looking I T guy but uh you know uh but again you know I will say that I know DT has you know triple their investments in tree trimming to help do a lot of that preventive maintenance so we don't have as many outages but you know only we can only go a certain far you know over the last couple years we have seen more major storms more frequently and more often than we have in our entire history of our company so you know for us we have to do something differently so you know uh as part of that starting in 2021 Improving Customer Communications - EFC Project um we want to put a new initiative in place to help improve the customer experience what we observed during that time was um you know we saw the most outages in 2021 the largest right in our systems you know we're on older technology right so we said hey we're going to come in we're going to upgrade our new systems and not just upgrade our systems and also improve the way we're communicating to customers so you know we designed our customer facing systems to improve the communications a lot of enhancements within our terminal systems right our back-end systems you know you hear some words like say Ami meters that's the meters that says hey you out of power and how much electrical uses you have and try to utilize some of that information and deployment of Key Systems you know we moved over from on-prem to to Azure platform and you'll hear about that today I I'll call this you know word whismo a lot and you'll hear that and it's you know the back end is on Azure platform and then also internal systems to scale up right so um again you know be able to handle these large storms you know at the peak period when customers need us most and so we can respond back to them uh proactively And Timely and like I said uh the last piece which we'll talk about a lot of the analytics side is understanding by building a monetary triage and measure and see where we're at how many customers are out of power or what is going on or do we have any issues are there any lagging or leading indicators so you know developing you know tools and process in place on top of the customers experiences understand what happened and where they're at with that in their outer's life cycle um I haven't done a presentation in 15 years so you know beg with me here as I flipped through my pages and my little notes um so this is our uh EFC you know we call it the error-free communications architecture you'll see here this is EFC Application Architecture kind of a high level diagram in the center here you'll see the outers tracker wismo otherwise known as wismo you'll hear me for that that is sitting on Azure um uh platform um and then that's our Central system that we tracked and all what we do is this is Central system to track all outages for a customer's perspective so you know our our backend system is our outage management system so as outages occur that's where customers that's where we do our jobs accrued through the jobs that says hey we have these outages they say the crews to restore those so our job is to take that information put into wismo and disseminate that information and then communicate out to the customer channels whether it's mobile app ivr outage map web notifications our CRM system we have an sap CRM system that's what our CRS use and then the data Lake right you know again a key piece you know for analytics and store net in there for all our data bit dashboards and Reporting again my focus will be talking about the wismo component of that because that is our single source of Truth for all outage information from a customer perspective whether the outer's been reported estimates how many outages what happened Restorations what it look like an outage map to be able to triage it so we're going to talk you know the rest of the pieces about that that area specifically um one of the things we first did when we first came out with this project says hey we got to do you know the zero free communications right and wismo is a back-end infrastructure it's all integration right so as you guys know anything that's back in uh just pure interfaces there's really no it's you know in the past it's a lot of manual testing right just one thing here on this end to come on this end and we want to do something different and smarter um we got to have something that's quicker easier more repeatable right and a higher quality so we decided as part of the initiative and you know talking to leaders like what do we do we could have picked a lot of things like J meter and things of that nature but decided to use your iPad that was fit for purpose right it was test automation um and and part of that is you know to to make sure we can spin something up quickly easily and manage that so we went from basically zero to 300 um test cases in one month I mean you know just with a few people um you know pose a dozens of testers taking weeks to go through there and those test cases is a push the button so you know our ability to scale up and and see what's going on and pass fail and be able to turn that around that high quality perspective was was great not just in addition to the tool I mean we had some a lot of good you know I called Nick wherever Nick said he's here he's standing in the back you know call them the secret sauce you know really sharp guy he understands the tools and and how these systems work um and then the 600 test case scenario so um along with it and we can mature that right so we've leveraged this information time and time again over and over it says reputable process um and one of the other things we did is like not just it wasn't just good enough to say hey we can repeat these test cases but able to run a load test saying hey if we had a million outages or a million transactions and not just from a customer reporting but you know if you go back to that old scale this old system there's a lot of interfaces coming in and out right out of status we got people you know pounding on our keyboard saying hey what is the latest status checking the map going where am I at you know the back end system saying is job updates and estimate updates and making sure that that volume can our our platform can support all that so and we leveraged like I said you know do iPad for the test automation to scale that up just to see to make sure the system was sustain that type of volume um you know not in addition to that so you know in moving us from uh you know agile to more of a devouts model so um and we truly are doing that so we are repeating that process and doing you know Sprints and two weeks iterations and part of our development and rolling out is regression testing all the test cases and incrementally improving those as we find new scenarios that may have failed we don't have many um which is great you know I think our team I give more credit to the team than than anything else to identify as and understand them how that information works together make sure we have test coverage for that and again and what's important about that is an improved customer experience right so making sure that hey when you're going to check your you know your notification matches what happens on the map when you look at it and the data is consistent um you don't have this hey why is this show one thing here and this says another thing here to make sure that those messages are are across the board and then uh as well um so again uh that's kind of the overall system so you know there's two aspects of this data quality and analytics right so I want to take one side as you know as we transition to the test automation which has been great you know a great success tool I'm so happy that we decided to do that as part of our overall process and and from a quality management perspective you know we're able to turn the work around as fast as we can be as as a result of that um and then on the analytics side right we know our understanding our out is in triaging and then our proof customer experience and what am I mean mean by that is understand the scope and scale of outage events whether you know we have this concept you'll hear me talk about Blue Sky versus storm right blue sky is a nice pretty day you know not many outages and we have these crazy storms that we end in February was like our second largest one ever in our history right where we call it a major storm it's all hands on deck and how do we manage to understand the scope and what's going on so um you know looking at that um the different dimensions of the information and then our customer experience is um you know what do we do to the customers how do we message the customers be able to see and go back and look going if there is a bad experience be able to triage that going okay this is what happened to customer we sent them you know they reported their outage we sent them the location or we close it out but then they open up another one again it's like oh no we did you know customers when we call it a Trouble behind trouble there's something else going on with that customer but we treat those guys differently but at least we can see that information from a single Source not that we didn't before right but it's just taking it to the next level of that data and information not just looking at from a job perspective but looking at an individual customer and and scale that up in a real-time perspective so that's one of the great things too that we kind of realize quickly as we kind of started this project was not just to test automation it's like oh we've got it we've got to build up analytics better to figure out what's going on because we have so much data now that's at our fingertips you know we need to develop you know power bi tools and and we use all tricks and other all these other pieces to understand that data because we have it um so a lot of the experience that way did we kind of go in knowing that we had to do that no but we quickly understood the need and the opportunity to be able to do that um to see that information so again I'm kind of going into some of the key and election requirements so some of the key requirements we kind of Key Analytics Requirements & Assets took a look at and our scientists um you know asking the common questions you know how many hours we have did the outers go through do we get estimates for a customer um you know how many hours do they have what is the time series in terms of information what is the current job status so one of the things we know I talk about this word of job is you know the job to complete somebody's restoration you can have 15 people on the job you have thousands of people and all those customers are in you know to be able to see that Association um in the life cycle uh life cycle what I mean is by you know for customer reported out is one two five times you know they may have gotten five enter you know five estimate updates why do they give five estimate updates our goal is to only have one so how do we you know what do we do wrong and how do we make it better um I'm exaggerated by the way that doesn't really happen a lot so I just want to say that I gotta be careful um and then what are some of the exceptions outliers right what are some of our error handling so we know that we're not perfect but be able to Quant you know identify and quantify it um something we didn't really leverage that today by utilizing these you know data analytics tool power bi and we use all trucks we can see what the scope and how to remediate those so you know it's an ongoing Evolution process to do those pieces and we will kind of go into some exceptions and what we kind of do to help with that and then what are some data quality issues um you know some of the things that we have is just you know bad addresses or missing addresses um you know uh inactive meters customers are trying to report on a house that has no active meter because they selected the wrong address because they live at you know Outer Drive versus West Outer Drive Outer Drive is one of our favorites I have Ali here that we always take a look at because Outer Drive snakes through and people just pick the first Outer Drive opposed to their house and uh it has an impact overall um and then again like I said we talked about the wismo those are some of our dashboards we have one thing we call the wismo orders how many orders or how many orders we have the jobs that are associated with those outages uh recessions dashboards and then um some of we do a comparison to make sure we're in sync with our back end system so um want to make sure we're not dropping messages or out of sync with how we're communicating to customers we things like you know storm mode tracking if we're you know we're in blue sky or storm because we do messaging differently so we want to track that and what we've done date and time so we have to take a look back and see what we sent to that customer and then we have this other concept called orphan orders these are outages that kind of get stuck in the process and usually they're typically data issues that causes some events so be able to identify those and get those resolved in a different process so we trapped those as well so it's kind of our we call them our orphans our leftover children that we have to kind of manually take care of and address Outage Status Dashboard um this is one of our outer stats dashboards so uh it's you know um when I started this whole project right not very heavily into dat analytics you know I understood reporting needs is essential to success and be able to measure it so you know be able to see this information in a single point of view and actually have these dashboards using power bi or other systems been great um and from a visual perspective so you know the team we have on here is like hey I can do this and that and I can plop you know the outages on the you know Michigan map and make it look pretty and see the time Dimensions uh so you know some of the things we've been able to do just from a visual perspective and data perspective you know we didn't have before because we didn't really leverage you know um you know power bi and some of the other things that were out there to take a look at this data so it's been I personally I think it's kind of cool but maybe you guys haven't yeah the data Lake you have to go back we saw everything in our data Lake yep um so again so here's like real-time open outage status so anytime we come and look here and take a look we can see the life cycle that you know other outages you know what the dim dimensions are how long they've been open how long they've been restored and we can actually search by um a person's house right and saying hey what happened with this you know look take a look at the data to be able to triage and see it's overall or just the accumulation right be able to slice it going hey we had a thousand outages you know today and this crazy storm went through this area and we jumped to 15 000 in an hour and we could be able to slice that information out how did we do let's say we did pretty well um and then again that's like the clusters of various outages so kind of taking a look at that little map you know it's just a little snapshot you can kind of see there's 44 humping outages and the map kind of sprinkled them all across um and again this is simulating this is our test environment um but but anyway it's kind of assimilate that information and then the bottom right side you can kind of see these multiple customer called 11 times but because it's test environment we can kind of see hey is there a problem there area with their outages with certain customers so we're able to see those and track those and figure out is it is it a data issue is something else going on is it us you know and trying to take a look at some of those those pieces and slice that out um storm dashboard so I told you uh you know we have this concept called Blue Sky and storm you know storms so when the storm happens it is a very much uh all hands on deck type of situation um where you know everywhere is their storm hats and you know whether you're in a field restoring stuff or in your I.T making sure the systems are scaled up to support and we have people watching and making sure that things are being addressed so one thing we had done is taking a look at the stat you know this information using the wismo as a back-end and the data Lake and putting a storm type of view on it so this this is a storm we had in August last year um it was again one of the craziest largest storms we had it's a big theme the last few years I don't know why but it has been it's been kind of shocking that's why we've had to scale up but you can kind of see here is like where we're at in the storm so we have this concept of some storm theme faces where we're at so you know at the ramp up period right so when the storm first comes through it says you know we go from you know a thousand outages to you know 371 000 outages right in just a matter of hours but able to see that scale and how did our systems have and then how we're doing this as we kind of go along our progress you know do we have estimates um are we restoring people what does that timeline look like as restaurant restore people or is the numbers trending down why isn't it is there a spike coming back up are we doing something wrong so we'll be able to see that insight and that's kind of what we utilize the storm dashboard for it's real time in a moment and something you know we've always done but this is taking the data from a different type of view and it's from a premise view opposed to looking at those jobs it's just the way we look at the data is differently and it's uh which I guess more detailed I guess to save more or less and then we can like I said time series you know what do we look at from previous storm events we're able to tie it back and we kind of did this little snatch out of the 2022 of in the bottom right corner going here's all these what we call Little storms and we have that big August storm you know it doesn't even scale it was just it was crazy crazy uh times but um but we had the information right and we can see what's going on and improve on that and that's just something that uh was part of our goals right or a few Communications be able to measure those pieces everything's out the daily no it's real time event so as the system comes through and updates events whether a customer reports and nowadays or again another statement update from the back end we updated lately you know it's uh Road time is real-time events you know we have activities going on we track everything that happens not everything we do with systems updates we give from the back-end systems results in a communication to customer it's just hey we you know assign the crew we don't you know do crew updates or we change it from one job to another job but that all that stuff is stored in there we we actually have everything go back and look at it previous new changes you went from outage to a Flickr right um to you know all the types of information um thanks for asking questions by the way standing up here keep talking I get nervous uh how does the admin dashboard so this is the second piece I talked about one was around the analytics and taking a look at these dashboards the other piece we decided to put into is like we have all this great information that's sitting in you know wismo and the data Lake and we need to provide this information not just for it but for our business partners right for sales service to figure out what happened right so there's many people that want this information so we'll be able to triage and see what that experience is so what we decided to do was develop a front-end call we call it the outage admin dashboard UI and on here you'll see is what what motor are we in if we're in blue sky or storm because as I mentioned before we measure customers differently and be able to switch that uh with that we see how many outages are a real quick Lance right it's just a summary of where we're at with the outage systems um and then the other piece is in here at the top kind of seeing you know I call it the counts but it's really how many outages we have open and how many jobs we have so just some additional information so we can kind of take a look at from a seamless summary with Alpha having to go into the data Lake it's it's actually open active information um the next two slides as well be able to triage so if you're out of power of calling in saying hey you know I've called you guys five times or you know what's going on or hey you know we saw this customer that you know never got a power status restoration so we'll go back in and look and see hey yep we got an outage here we send a customer notification we see they called in right check status and we've updated the estimate three times but you can actually see the history not just for the current outage event but but all of them so again not having to you know if everybody's an I.T the business is always asking questions about the business able to leverage this this is one of those data quality improving the customer experience not just external customers but our business partners too and then um you know managing our storms and our messages and what we say and do to the customer so a single point of view right so if you look at my other slides we talked about the ivr the web mobile and the map all the messages that go to those channels it's managers in a single place so they're consistent with across the board in a timely fashion so if we wanted to change a message that you know we change in one place and it gets propagated down Data Quality Identification & Remediation um so identifying data quality and Remediation so this slide here you know our left hand side here is our exceptions dashboard a lot of funny codes you know bad reads 95 our calls you know but it's our internal languages but basically is you know these are things we identified over time wise we're testing or as we're in production that says hey we want to identify what these outages you know what these things are so we can fix them right for the next time for it doesn't happen again so both of them dedication also to ensure from a quality perspective going forward that we don't have these problems and a lot of them are usually Source data issues like incorrect data um you know that address problem that I tell you about that you know Outer Drive is gives us a headache sometimes or incorrect lat long so we'll get to that in a minute uh one of the things that we also worked on was the outage map and making sure the data quality on the map is actually accurate um and I'll show you some of the daggers and Spears and arrows that stab me over time until we got that down um and then a data issues right out of status updates maybe the interface went down or while we're not getting out of status updates what's taking so long so we don't see that tracking um you know mismatches are just what we call a Trouble behind trouble where customers says hey we thought you restored power they called back in and says I'm so out of power to be able to track those and saying hey what's going on with this and then system equipment right faulty meters it happens or we're not getting a good connection with the meters it's over you know it's all outside of the thresholds or system outages maybe one of the other interfaces went down between the systems to be able to see that and scalability right so we talked a lot about you know scaling out can the system handle the volumes right so as we all 19 know when you know we have a lot of stuff flooding the system sometimes things slow down and that scalability I swear to monitor those things and saying hey what's going on Outage Map Data Quality Problems um kind of moving on to the outers map so we talked about that data quality issue so this is I I joke so we're working on this new outage map again um putting it in so when we're going our testing then the new map we used to the old one the data is different um it's at a much granier level we're at a customer house level and the old map USB higher level but that's besides the fun but we're trying to take a look at this data we're like oh my we hooked it up to the first map we started putting this information on I'm like where are these need what is these like a needle in my eye I'm like getting stammed like I can't stand this every day so I was talking to Kristen and I think I'm like guys this is crazy we can't show a customer polygon that's you know across Three Counties in five cities right so what do we do you know we got two million records how do I identify and fix all this right what they should be looking like these little nice little pretty pictures on the left-hand side which are great right very this is the neighborhood the houses yep kind of makes sense um and what really it is on the right hand side is that needle that far end it says this house that long or the equipment hierarchy is doesn't fit so it extends this whole piece so we needed to have a process in place to figure it out so of course you know 1-800 Scott Ollie and they're like hey I think we can do something right so they went back and about a week later it says you know I think there's a couple things we can do to fix this and what they did was Outage Map Data Quality Resolution um uh you know use alteryx um I don't know if you guys know I learned this alter YX YX is XY positions for lat Longs FYI I just thought that was kind of cool learned that too this year um to be able to identify with those processes are so they were able to load all 2.4 million I would say Ollie here this is Ali guys so thank you Avi I was able to load all those meters uh into all tracks and spin it up and says here's all the lant Longs that we have and here's what Google says and be able to see what the differences were right so our example on the side I said here's this outage that was in Ann Arbor but I had one meter because back in the day they fixed a meter and it got signed way out in bad ax so if we hit an outage it was 97 miles away so that you know and that needle I was showing you that's what happened so we were able to identify those outliers um and the report you really can't see is this dashboard um and this for it says outlier distance when we first got there we had outages with light longs it was in Antarctica and on a lake and Hamburg Germany opposed to Hamburg Michigan but where to fix all those things quickly and uh efficiently so took all those discrepancies and said hey we automated it we fed it through Google to make sure because the back end map was tied to Google it updated all those so instead of being miles away we're within thousands of feet of the of a person's house so high quality updated hundreds of thousands of Records uh to be reflective on that and then the other piece too was uh the second part was um you know you hear this word transform right everybody's hooked up to a transfer of your house is in your area so it's important how we kind of mapped them the outage map with that as well um so we might want to make sure that those meters were within that Transformer so these outliers was the other aspect so we can actually fix that information so you know proactively so we went live for the map we were identified you know by utilizing these tools in this dashboard not not just one time we do this ongoing right as part of our process to make sure as we update our you know we call it delivery nodes and to the map to present that information that's accurate um huge huge success we would not have been able to do it because I went from like getting stabbed with needles in my eyes to tears of joy I mean the big joke in our team but you know can you imagine trying to figure out two million records and how you're going to update that manually and you know through a quick couple quick tools all tricks you know kind of spin it all through not just data wise but also give it gave a visual view of what those outliers were I'm a couple slides behind hold on a second um and again like I said um you know you know the combination of a lot of these tools between um uh you know power bi all tricks right the automation tools Right Moving us from the you know our on-prem system to a new Azure platform and using the data Lake to see this information really it brought us from a community customer perspective or air for information to the next level um and be able to scale up and support and know our needs like I said as you guys know good or bad these crazy storms are happening more often right just had another one in February which is on our second or third largest in 50 years or something like that but those words come more and more often we've got to be prepared to support that and someone is back-end systems we did really move towards that right and be able to see that had that insight into a customer of what happened from a single point of view um there's been been a phenomenal tool so again like I said uh kind of recap you know you know uh you know when we Services & Software Partners started this adventure you know was you know I have see the back here we know we decided to bring wood on board and some of the resources they really strong in it automation analytics tools um you know you like I said use all tricks I couldn't figure out the map issues if I didn't have the all trucks didn't even know what that meant until you know these guys told me that hey we can spin this up and take the visualization um uipath that's kind of where we started I just wanted to kind of do something different from a development perspective and get test automation as part of the process move to devops and and the power bi right so having that data in there and have a single point of view and be able to measure manage and quantify what is going on with the customer experience so it's been a fun project I mean you know these are some of the simple slides but um I had that type of insight I've been working at DT for a couple years I should say a lot of years and you know just taking that information to Next Level has been it's been quite experience and and really enjoyable and and I would say a key customer success for us kind of moving forward um you know we've you know this is just one small piece of our air free communications and Outreach management system we've been upgrading a lot of our systems and you know I'm just you know focused on one slice of of what we have done um but anyways any questions yeah I would say that would be the the biggest piece we usually lies for it is from that perspective for data for the data quality is really looking at from a uh you know lat long geospatial type of stuff you guys played with Microsoft review at all no we haven't no we haven't done any of that type of stuff like I said I've no I know that stuff is out there but our team has been kind of you know there's a lot of reporting tools but yeah yeah yeah no that's yeah quick dirty quick and fast yep those four sources um well the meter stuff is in our backend OMS system outage management system so they do collect all the information from a meter hey you know um this is not my expertise by the way but that type of stuff but from a customer side as as they report outages or checks as we do collect that stuff within within our system or have webmetrics oh over from Ami perspective yeah the Ami meters though typically so I think it's 64. 30 yeah something like that about 30 percent yeah during the big storm yep yep I didn't want to yeah I'm like I can remember the Quake number so yeah yep uh anybody else yeah well I would say there's a whole team that kind of goes behind it say high level is you know they they as a group they kind of take a look at those every day but um they call it plan a day and I think typically it's usually by large South edges but you know uh yeah pretty much yeah I mean obviously some are obvious you know you figure the outage there so they go substation down they kind of know and work on those big pieces first right so I uh they are looking at some of that stuff I've been not involved in that side of the business to to take a look at I'm more on the customer on the customer side of that opposed to looking at the analytics and metering and using some AI you know predicting the outages and saying you know this is what I think is really out I know they are working on some of that but I appreciate the attendance here this is great this is I feel I was nervous you guys were but again I'd like I said thanks for attending like I said it's been a great experience and uh appreciate your time oh [Applause]