This was an old forgotten grave. Ghost hunters come closer and everyone felt very sad and angry presence.
Intrigued, they came closer, and everyone seen in light of full moon a very scary epitaph on the tombstone, covered with old cracks:
"He deployed to Production at Friday 13"
it
This morning, I took my wife to the hospital for routine blood tests that had been scheduled for some time. Everything was going smoothly: check-in, number, waiting room. Suddenly, everything came to a halt and shut down. I was connected to the hospital’s public Wi-Fi and noticed that my connection also went down.
Having managed a couple of similar facilities, I immediately understood what had happened. I saw the staff panicking and calling the technicians, but they quickly reorganized within 10 minutes. They managed to process everyone who already had a number and then proceeded with the others in the order of their arrival. Despite the ten-minute delay (even though people started complaining right away), they were extremely efficient.
I later confirmed that the entire booking, check-in, and queue system is “in the cloud.” The hospital experienced a connectivity interruption, and all related services stopped. The staff no longer had access to anything, so a technician sent the lists to a manager via another channel, and everything resumed manually.
For years, I’ve insisted that certain things MUST be local. The healthcare facilities I manage have all the necessary systems for the operation of the facility internally, including patient records. External services like websites, emails, etc., are secondary.
Everything essential must always be accessible locally and, in special cases, it should be possible to physically access the servers and connect directly to them, bypassing any network/switch failures.
There has been only one interruption in the past, due to human error. Today, we have redundant servers (not HA on virtualizers, but two machines running the same software with replicated databases - on separate power lines) so such an issue shouldn’t happen anymore.
Not everything can be anticipated, but history is a great teacher. The Internet connection will eventually be interrupted :-)
When it comes to the health and survival of people, there are no compromises.
#IT #Internet #Networking #Outage #Health #HA #Cloud #CloudComputing #OwnYourData
The slides, the video, and the text behind my presentation at EuroBSDCon 2024 - 'Why and how we're migrating many of our servers from Linux to the BSDs.'
https://it-notes.dragas.net/2024/10/03/i-solve-problems-eurobsdcon/
#ITNotes #FreeBSD #OpenBSD #NetBSD #RunBSD #IT #SysAdmin #EuroBSDCon #EBC24 #EuroBSDCon24 #EuroBSDCon2024 #NoteHUB
| ̄ ̄ ̄ ̄ ̄ ̄ ̄ ̄ ̄ ̄ ̄ ̄ ̄ ̄ ̄ ̄ ̄|
| Don't Push To Production On Friday |
|_________________|
\ (•◡•) /
\ /
——
| |
|_ |_
#productivity #product #push #ascii #it #asciiart #friday #dontpush #dev #reallife #hotfix #git #codejokes #nerdjoke
Average YouTuber: "My mobile carrier is terrible today; I'm only getting 200 Mbps! How am I supposed to survive?"
Me: "Hey, I get 20 Mbps on my phone's hotspot! I can work just fine!"
"New servers are less energy-intensive." Really? 🤔
This is said so often that it could almost be taken as an absolute truth.
By extension, it is often said that each new generation of hardware consumes less energy than the previous one.
Manufacturers communicate pretty often about this, including HPE (1), Nvidia (2), etc..
A 🧵