tag:onfleet.statuspage.io,2005:/historyOnfleet Status - Incident History2024-03-28T17:06:37-07:00Onfleettag:onfleet.statuspage.io,2005:Incident/201702782024-03-05T14:10:53-08:002024-03-05T14:10:53-08:00Network Connectivity in California<p><small>Mar <var data-var='date'> 5</var>, <var data-var='time'>14:10</var> PST</small><br><strong>Resolved</strong> - Per AWS, between 11:43 AM and 1:30 PM PST some customers in California may have experienced connectivity issues between their ISP and AWS destinations. Onfleet was able to reproduce the issue from San Francisco with Comcast connections but not ATT connections.</p><p><small>Mar <var data-var='date'> 5</var>, <var data-var='time'>13:24</var> PST</small><br><strong>Investigating</strong> - AWS reports connectivity issues for some customers in California. Onfleet has received customer reports of problems connecting to the service.</p>tag:onfleet.statuspage.io,2005:Incident/198940532024-02-02T08:00:16-08:002024-02-02T08:00:16-08:00Issues with Task Exports<p><small>Feb <var data-var='date'> 2</var>, <var data-var='time'>08:00</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Feb <var data-var='date'> 2</var>, <var data-var='time'>07:37</var> PST</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Feb <var data-var='date'> 2</var>, <var data-var='time'>07:23</var> PST</small><br><strong>Investigating</strong> - We are currently seeing issues related to task exports email being sent out.</p>tag:onfleet.statuspage.io,2005:Incident/196523872024-01-08T18:58:10-08:002024-01-08T18:58:10-08:00Service Degraded<p><small>Jan <var data-var='date'> 8</var>, <var data-var='time'>18:58</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jan <var data-var='date'> 8</var>, <var data-var='time'>18:55</var> PST</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Jan <var data-var='date'> 8</var>, <var data-var='time'>18:44</var> PST</small><br><strong>Identified</strong> - An error in code led to an unintended functionality change. The problem release is being rolled back.</p>tag:onfleet.statuspage.io,2005:Incident/196195872024-01-05T01:45:10-08:002024-01-05T01:45:10-08:00Service Degraded<p><small>Jan <var data-var='date'> 5</var>, <var data-var='time'>01:45</var> PST</small><br><strong>Resolved</strong> - Between 7:25 PM January 4th and 1:43 AM January 5th tasks appeared out of order on the dashboard side panel. The issue has been resolved.</p>tag:onfleet.statuspage.io,2005:Incident/194779482023-12-19T01:22:15-08:002023-12-19T01:22:15-08:00System Unavailable<p><small>Dec <var data-var='date'>19</var>, <var data-var='time'>01:22</var> PST</small><br><strong>Resolved</strong> - All changes were reverted, and the service is now fully operational.</p><p><small>Dec <var data-var='date'>19</var>, <var data-var='time'>01:06</var> PST</small><br><strong>Identified</strong> - Our team has successfully identified and reverted the changes that caused the disruption. Services are being restored to full functionality.</p><p><small>Dec <var data-var='date'>19</var>, <var data-var='time'>01:04</var> PST</small><br><strong>Investigating</strong> - We are currently addressing an unexpected disruption that occurred during the deployment/upgrade process. Our team is actively working to resolve the issue.</p>tag:onfleet.statuspage.io,2005:Incident/190480292023-11-07T11:11:49-08:002023-11-07T11:11:49-08:00Telephony - US shared toll-free number degradation<p><small>Nov <var data-var='date'> 7</var>, <var data-var='time'>11:11</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Nov <var data-var='date'> 7</var>, <var data-var='time'>10:50</var> PST</small><br><strong>Monitoring</strong> - We have worked with our telephony provider to resolve the issue and are now monitoring traffic from the shared high throughput toll-free phone number.</p><p><small>Nov <var data-var='date'> 7</var>, <var data-var='time'>09:08</var> PST</small><br><strong>Investigating</strong> - We are investigating issues related to the shared high throughput US toll-free phone number used for notifications and call anonymization that is affecting some customers.</p>tag:onfleet.statuspage.io,2005:Incident/180657582023-08-06T01:29:28-07:002023-08-06T01:29:28-07:00System unavailable<p><small>Aug <var data-var='date'> 6</var>, <var data-var='time'>01:29</var> PDT</small><br><strong>Resolved</strong> - Our API and Dashboard experienced intermittent errors and elevated response times between 1:01AM to 1:08AM PDT. We restarted all affected servers and API requests resumed processing normally.</p>tag:onfleet.statuspage.io,2005:Incident/179172572023-07-20T16:30:00-07:002023-07-20T17:12:46-07:00Service degraded<p><small>Jul <var data-var='date'>20</var>, <var data-var='time'>16:30</var> PDT</small><br><strong>Resolved</strong> - Our dashboard experienced intermittent errors between 4:32PM PDT and 4:48PM PDT. Some users may have experienced issues loading the dashboard and tracking pages.</p>tag:onfleet.statuspage.io,2005:Incident/178652412023-07-15T13:30:00-07:002023-07-15T14:55:16-07:00System unavailable<p><small>Jul <var data-var='date'>15</var>, <var data-var='time'>13:30</var> PDT</small><br><strong>Resolved</strong> - Our API and Dashboard experienced intermittent errors and elevated response times between 1:33PM and 1:46PM PDT. We restarted all affected servers and API requests resumed processing normally.</p>tag:onfleet.statuspage.io,2005:Incident/177936852023-07-02T14:00:00-07:002023-07-07T13:34:35-07:00Service degraded<p><small>Jul <var data-var='date'> 2</var>, <var data-var='time'>14:00</var> PDT</small><br><strong>Resolved</strong> - One of the web proxy instances failed causing transient errors for API and dashboard services from 1:57PM PDT to 2:04PM PDT.</p>tag:onfleet.statuspage.io,2005:Incident/171239842023-05-02T03:30:00-07:002023-05-02T12:53:23-07:00System unavailable<p><small>May <var data-var='date'> 2</var>, <var data-var='time'>03:30</var> PDT</small><br><strong>Resolved</strong> - Our API experienced intermittent errors and elevated response times between 03:14 and 03:30 PDT. We restarted all affected servers and API requests resumed processing normally.</p>tag:onfleet.statuspage.io,2005:Incident/169884472023-04-27T20:27:51-07:002023-04-27T20:27:51-07:00Issues performing imports<p><small>Apr <var data-var='date'>27</var>, <var data-var='time'>20:27</var> PDT</small><br><strong>Resolved</strong> - The incident has been resolved.</p><p><small>Apr <var data-var='date'>27</var>, <var data-var='time'>19:52</var> PDT</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Apr <var data-var='date'>27</var>, <var data-var='time'>19:22</var> PDT</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Apr <var data-var='date'>27</var>, <var data-var='time'>19:09</var> PDT</small><br><strong>Investigating</strong> - We are investigating issues experienced by some users in performing imports in the dashboard.</p>tag:onfleet.statuspage.io,2005:Incident/166316682023-03-23T22:00:00-07:002023-04-03T14:57:08-07:00System Unavailable<p><small>Mar <var data-var='date'>23</var>, <var data-var='time'>22:00</var> PDT</small><br><strong>Resolved</strong> - We identified elevated response times on our API between 10:00PM - 10:12PM PDT. We have restarted all affected servers and API requests started being processed normally.</p>tag:onfleet.statuspage.io,2005:Incident/164227382023-03-18T03:40:11-07:002023-03-18T03:40:11-07:00Networking infrastructure maintenance<p><small>Mar <var data-var='date'>18</var>, <var data-var='time'>03:40</var> PDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Mar <var data-var='date'>18</var>, <var data-var='time'>03:13</var> PDT</small><br><strong>Verifying</strong> - Verification is currently underway for the maintenance items.</p><p><small>Mar <var data-var='date'>18</var>, <var data-var='time'>03:00</var> PDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Mar <var data-var='date'>10</var>, <var data-var='time'>17:39</var> PST</small><br><strong>Update</strong> - We will be undergoing scheduled maintenance during this time.</p><p><small>Mar <var data-var='date'>18</var>, <var data-var='time'>03:00</var> PDT</small><br><strong>Scheduled</strong> - We will be updating our networking infrastructure on Saturday, March 18th 3:00AM PDT / 6:00AM EDT / 10:00AM UTC.<br />We expect a very brief period of service unavailability while these operations are taking place.</p>tag:onfleet.statuspage.io,2005:Incident/164217802023-03-09T01:00:00-08:002023-03-10T08:23:20-08:00System Unavailable<p><small>Mar <var data-var='date'> 9</var>, <var data-var='time'>01:00</var> PST</small><br><strong>Resolved</strong> - The service is unavailable / inaccessible.</p>tag:onfleet.statuspage.io,2005:Incident/163565492023-03-06T10:51:16-08:002023-03-06T10:51:16-08:00Email Delivery Issues Affecting Exports and Other Transactions<p><small>Mar <var data-var='date'> 6</var>, <var data-var='time'>10:51</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'> 6</var>, <var data-var='time'>10:35</var> PST</small><br><strong>Monitoring</strong> - Exports from the dashboard are starting to deliver normally. Continuing to monitor until the transactional email provider has declared all clear.</p><p><small>Mar <var data-var='date'> 6</var>, <var data-var='time'>10:01</var> PST</small><br><strong>Identified</strong> - The transactional email provider has confirmed the issue and is investigating it on their end.</p><p><small>Mar <var data-var='date'> 6</var>, <var data-var='time'>09:35</var> PST</small><br><strong>Investigating</strong> - We are investigating issues with email delivery which appear to stem from our transactional email provider</p>tag:onfleet.statuspage.io,2005:Incident/163349542023-03-02T17:30:00-08:002023-03-03T17:03:33-08:00Geo-coding errors preventing task creation in certain regions<p><small>Mar <var data-var='date'> 2</var>, <var data-var='time'>17:30</var> PST</small><br><strong>Resolved</strong> - Errors in geo-coding temporarily prevented tasks from being created, especially in regions of the world with inaccurate or incomplete mapping data.</p>tag:onfleet.statuspage.io,2005:Incident/160467672023-02-06T16:38:03-08:002023-02-06T16:38:03-08:00Route Optimization / Auto-dispatch experiencing increased failure rates.<p><small>Feb <var data-var='date'> 6</var>, <var data-var='time'>16:38</var> PST</small><br><strong>Resolved</strong> - Optimization services are fully operational.</p><p><small>Feb <var data-var='date'> 6</var>, <var data-var='time'>15:05</var> PST</small><br><strong>Monitoring</strong> - We no longer see elevated failure rates from the route optimization provider. We will continue to monitor the situation.</p><p><small>Feb <var data-var='date'> 6</var>, <var data-var='time'>15:02</var> PST</small><br><strong>Identified</strong> - We have detected an elevated failure rate from our route optimization provider and are in contact with their support team. They are working on the issue but do not have an ETA.</p>tag:onfleet.statuspage.io,2005:Incident/159535052023-02-04T00:41:03-08:002023-02-04T00:41:03-08:00Networking infrastructure maintenance<p><small>Feb <var data-var='date'> 4</var>, <var data-var='time'>00:41</var> PST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Feb <var data-var='date'> 4</var>, <var data-var='time'>00:40</var> PST</small><br><strong>Update</strong> - All operations have now been successfully completed.</p><p><small>Feb <var data-var='date'> 4</var>, <var data-var='time'>00:31</var> PST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jan <var data-var='date'>27</var>, <var data-var='time'>08:57</var> PST</small><br><strong>Scheduled</strong> - We will be performing some critical maintenance operations on our servers responsible for our networking infrastructure, as mandated by our cloud hosting partner.
<br />
<br />We expect a very brief period of service unavailability while these operations are taking place.</p>tag:onfleet.statuspage.io,2005:Incident/160706662023-01-27T01:30:00-08:002023-02-08T19:51:00-08:00Unavailable Driver Location Data<p><small>Jan <var data-var='date'>27</var>, <var data-var='time'>01:30</var> PST</small><br><strong>Resolved</strong> - We are currently investigating this issue.</p>tag:onfleet.statuspage.io,2005:Incident/157744872023-01-05T08:35:52-08:002023-01-05T08:35:52-08:00Errors sending email<p><small>Jan <var data-var='date'> 5</var>, <var data-var='time'>08:35</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jan <var data-var='date'> 4</var>, <var data-var='time'>19:03</var> PST</small><br><strong>Monitoring</strong> - Mailchamp Transactional has reported that the issue is resolved. Onfleet is monitoring the recovery.</p><p><small>Jan <var data-var='date'> 4</var>, <var data-var='time'>19:00</var> PST</small><br><strong>Update</strong> - Mailchimp Transactional error rates are decreasing.</p><p><small>Jan <var data-var='date'> 4</var>, <var data-var='time'>18:00</var> PST</small><br><strong>Update</strong> - Mailchimp Transactional is continuing their investigation to resolve connection issues.</p><p><small>Jan <var data-var='date'> 4</var>, <var data-var='time'>16:13</var> PST</small><br><strong>Identified</strong> - Onfleet has identified an upstream issue with our Mailchimp email provider. Mailchimp is working to resolve the issue.</p><p><small>Jan <var data-var='date'> 4</var>, <var data-var='time'>15:57</var> PST</small><br><strong>Investigating</strong> - We are currently investigating the issue.</p>tag:onfleet.statuspage.io,2005:Incident/144924202022-12-06T04:23:38-08:002022-12-06T04:23:38-08:00Increased API error rate<p><small>Dec <var data-var='date'> 6</var>, <var data-var='time'>04:23</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Dec <var data-var='date'> 6</var>, <var data-var='time'>04:20</var> PST</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Dec <var data-var='date'> 6</var>, <var data-var='time'>04:18</var> PST</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Dec <var data-var='date'> 6</var>, <var data-var='time'>04:14</var> PST</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:onfleet.statuspage.io,2005:Incident/126817162022-11-01T04:18:05-07:002022-11-07T17:11:10-08:00Export processing delayed<p><small>Nov <var data-var='date'> 1</var>, <var data-var='time'>04:18</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Nov <var data-var='date'> 1</var>, <var data-var='time'>04:11</var> PDT</small><br><strong>Monitoring</strong> - An issue with export caused processing to be delayed.<br /><br />Previously delayed exports have now been processed.<br /><br />We are currently monitoring.</p>tag:onfleet.statuspage.io,2005:Incident/112066492022-09-21T09:11:27-07:002022-09-21T09:11:27-07:00Increased error rates for routing functions<p><small>Sep <var data-var='date'>21</var>, <var data-var='time'>09:11</var> PDT</small><br><strong>Resolved</strong> - Routing has been stable since 8:42 AM PDT.</p><p><small>Sep <var data-var='date'>21</var>, <var data-var='time'>08:25</var> PDT</small><br><strong>Monitoring</strong> - The routing functions are working correctly. We're going to continue monitoring and stay in touch with the upstream provider.</p><p><small>Sep <var data-var='date'>21</var>, <var data-var='time'>05:58</var> PDT</small><br><strong>Investigating</strong> - We are working with an upstream provider to determine to cause of this issue.</p>tag:onfleet.statuspage.io,2005:Incident/112007992022-09-20T21:35:45-07:002022-09-20T21:35:45-07:00Increased error rates for routing functions<p><small>Sep <var data-var='date'>20</var>, <var data-var='time'>21:35</var> PDT</small><br><strong>Resolved</strong> - Routing has been stable since 19:20 PDT.</p><p><small>Sep <var data-var='date'>20</var>, <var data-var='time'>19:32</var> PDT</small><br><strong>Investigating</strong> - We are working with an upstream provider to determine to cause of this issue.</p>