Announcements

EarlG

(21,947 posts) Wed Feb 28, 2024, 08:39 PM Feb 28

273

IMPORTANT: Today's update failed and we have had to restore a backup, but we did not lose data [View all]

As most of you know, today we attempted to update some of our software to a new version. We first performed the update on our test site, and after observing that everything was running smoothly, we updated the main site. It was at this point that we ran into issues, because the main site slowed to a crawl.

After investigating, we discovered that the latest version of the software involves some significant -- and unanticipated -- technical changes to the way DU works behind the scenes. Unfortunately these changes were not included in the documentation because, as we discovered too late, they would "only affect a handful of users." Yup, we are one of those users. This means we have to resolve this particular issue before we can continue with the update.

In order to bring the site back we had to restore a backup, but the good news is that because the backup was taken right before the update happened, we did not have to go back to an older version, and we were able to recover everything.

We are very sorry about the service interruption which, unfortunately, ended up being for nothing today. The good news is that we are now back to normal. We are back in our original configuration, so things should work correctly as of now. We expect that there will be some initial slowness as the restoration process continues -- you may notice it when first arriving at the site, but it should clear up fairly quickly.

We do still need to perform this software update, but we are now well aware of these specific issues which caused our last attempt to fail. We will not be attempting this update again immediately, but it should happens sometime within the next few weeks, and we are optimistic that our next attempt will be much more successful.

Once again, I apologize for today's downtime. Although we knew that this would be a challenging update, it's rare for us to run into a significant issue like this, so we're really disappointed that we weren't able to get everything squared away as intended today.

I'll update you again when we have more information about our next attempt to perform the update.

Thanks for your patience,

EarlG

78 replies

= new reply since forum marked as read

Highlight:

IMPORTANT: Today's update failed and we have had to restore a backup, but we did not lose data [View all] EarlG Feb 28 OP

No harm in trying and failing. ffr Feb 28 #1

Thank you and Elad for your great work! Wild blueberry Feb 28 #2

I'm very sorry the update failed, but glad to see you back, and without any data lost. highplainsdem Feb 28 #3

No problem. A suggestion perhaps. Schedule your updates late at night. OAITW r.2.0 Feb 28 #4

They addressed that in their notice yesterday rurallib Feb 28 #8

Did not read closely, my bad. Suggestion removed. OAITW r.2.0 Feb 28 #12

Any way to blame it on Mike Johnson and get the RNC to pay for our therapy? Hermit-The-Prog Feb 28 #5

Ha! Or at least the senseandsensibility Feb 28 #17

It's clearly Joe Biden's fault Ray Bruns Feb 29 #72

Well, it's good you chose a slow news day. jls4561 Feb 28 #6

At least you picked a slow news day. NT mahatmakanejeeves Feb 28 #7

the site being down was way too reminiscent of 2016. SleeplessinSoCal Feb 28 #16

Thanks for all that you do! Think. Again. Feb 28 #9

Got to say - been there, done that rurallib Feb 28 #10

My sentiments exactly. Gidney N Cloyd Feb 28 #42

Good luck! AltairIV Feb 28 #11

Create a restore point DoBW Feb 28 #13

Never let a computer know you are depending on it. n/t dobleremolque Feb 28 #14

Thanks for the update and I am ready for whenever your next shot at it is-n/t marked50 Feb 28 #15

Keep up the good work. It will happen! GreenWave Feb 28 #18

No apology needed, but thanks just the same! ShazzieB Feb 28 #19

Thank you for all you do! Hope22 Feb 28 #20

The joys of MIS... dlk Feb 28 #21

I'm just happy that you are up and running. live love laugh Feb 28 #22

Bummer... 2naSalit Feb 28 #23

Failure is important. madaboutharry Feb 28 #24

Been there done that slightlv Feb 28 #25

At least it wasn't a Neuralink patient you were working on! True Dough Feb 28 #26

Almost felt like I was going through withdrawal or something. calimary Feb 28 #27

No worries MustLoveBeagles Feb 28 #28

So glad we are back online! I'm sorry that you had a more Tanuki Feb 28 #29

As someone who works in IT, I fully understand! CousinIT Feb 28 #30

Thanks for all you do. virgdem Feb 28 #31

Thank you & Elad for All Your Cha Feb 28 #32

appreciate all you do... nt WarGamer Feb 28 #33

Thanks for trying malaise Feb 28 #34

Thank you for all you do and I also have a clean apartment, extra clean...LOL. SheilaAnn Feb 28 #35

It's cool. Happens all the time deep in software. Carry on. . . . nt Bernardo de La Paz Feb 28 #36

No worries. Stuff happens. LoisB Feb 28 #37

I know you guys were like BumRushDaShow Feb 28 #38

No need to apologize. You guys are still the best. EarnestPutz Feb 28 #39

Thank you for the update on the update that didn't update but will update!! chowder66 Feb 28 #40

Don't... orangecrush Feb 28 #41

No need for apologies. UTUSN Feb 28 #43

No worries. Deeply appreciate all you do for us and du. cbabe Feb 28 #44

I AM OUTRAGED!!!111one!! AnrothElf Feb 28 #45

such is the natural cussedness of things OldSWODog Feb 28 #46

To EarlG and Elad ... we understand and we'll get through this ... Rome wasn't built in a day, or even in a year FakeNoose Feb 28 #47

Thanks for all you do both visible and behind the scenes Golden Raisin Feb 28 #48

Won't work on my desk top computer Liberal In Texas Feb 28 #49

As a software developer, this failure was the best possible failure. eggplant Feb 28 #50

I know that was frustrating, but it's reassuring Pinback Feb 28 #51

No apologies necessary!! LoveTheDU Feb 28 #52

Thanks for doing this for us. MuseRider Feb 28 #53

I was getting worried; glad to see DU back. In case you didn't know, I was up at 4:00 AM ET when Backseat Driver Feb 28 #54

Situation Normal All Fouled Up (SNAFU) Jacson6 Feb 28 #55

Not a problem. PoindexterOglethorpe Feb 28 #56

Fun, fun, fun! Almost anyone who's worked in a company w computers knows there might be some downtime, somewhen... electric_blue68 Feb 28 #57

No worries mvd Feb 28 #58

I don't think anyone expects you to be clairvoyant. soldierant Feb 28 #59

Thanks for what you tried to do. greatauntoftriplets Feb 28 #60

Thanks for all you do to keep the site AKwannabe Feb 28 #61

Thanks for protecting our data. sinkingfeeling Feb 28 #62

I will admit that after the first couple of hours, I started having flashbacks niyad Feb 29 #63

Stuff happens. I work in IT, so I get it. Kennah Feb 29 #64

We're on your side.Do what needs to be done. We have have faith in y'all. marble falls Feb 29 #65

Them new fangled computer things sure are complicated, aren't they? Aussie105 Feb 29 #66

Absence makes the heart grow fonder. tazkcmo Feb 29 #67

Noticed a lot of missing Avatars today KS Toronado Feb 29 #68

I used to hate Windows updates Klondike Kat Feb 29 #69

Have you tried turning it off and then back on? twodogsbarking Feb 29 #70

No Worries EarlG.... Mr. Mustard 2023 Feb 29 #71

DU will not be available on February 30 all day Jerry2144 Feb 29 #73

I've noted a large number of user's avatars aren't displaying. n/t SpankMe Feb 29 #74

Thanks EarlG for the efforts and the update sdfernando Feb 29 #75

Could you try and pick a day for the update when SCOTUS isn't selling our Democracy in a fire sale to protect the...... usaf-vet Feb 29 #76

friend of mine said that failure is your friend . dont be too harsh on yourself. dont do any good. AllaN01Bear Feb 29 #77

Will our journals be coming back? Rocknation Mar 11 #78