server4 outage

Jeremy Howard · 14 Nov 2002, 02:57 AM

server4 has a database corruption problem. It will be down for a while--no ETA yet. No mail will be lost--it's being queued.

More info as we have it...

chicory_patch · 14 Nov 2002, 03:05 AM

Jeremy,

How can we find out which server we are on? Besides one day having no access because that server is down.

**DBH** · 14 Nov 2002, 03:10 AM

Open one of your messages and select Show full header next to the word Options in the top left corner. Look at the 2nd line that says 'Received:' ... somewhere in that line, you should see server5 . fastmail . fm (server5 is the server I'm on, at least)

Hope this helps!

Dave

punchcutter · 14 Nov 2002, 03:17 AM

Do I understand correctly from this exchange that
each user is assigned permanently to a given server, rather
than being assigned on a session-by-session
(or connection-by-connection) basis?

Jeremy Howard · 14 Nov 2002, 03:27 AM

Correct--your mail has to live somewhere.

There are multiple servers for 'stateless' stuff however (e.g. SMTP).

m5comp · 14 Nov 2002, 03:28 AM

Why can't server 4 users such as myself be switched temporarily to another server until the problem is fixed?

Jeremy Howard · 14 Nov 2002, 03:29 AM

I'm guessing that this will take 30 minutes to resolve at this stage.

DarioMor · 14 Nov 2002, 04:03 AM

Only curiosity... but reading the full header makes me more confused. In wich server am I?

Received: from www.fastmail.fm (server1.internal [10.202.2.132])
by server3.fastmail.fm (Cyrus v2.1.9) with LMTP; Tue, 12 Nov 2002 12:19:42 -0600
X-Sieve: CMU Sieve 2.2
Received: from server4.fastmail.fm (server4.internal [10.202.2.135])
by www.fastmail.fm (Cyrus v2.1.9) with LMTP; Tue, 12 Nov 2002 12:19:43 -0600
Received: from n13.grp.scd.yahoo.com (localhost [127.0.0.1])
by server4.fastmail.fm (Postfix) with SMTP id 39C242FD95

Jeremy Howard · 14 Nov 2002, 04:05 AM

The 'by' of the first Received line is your server (server3).

Jeremy Howard · 14 Nov 2002, 04:06 AM

My first guess was wrong, sorry. We're getting there, but the rebuild is taking longer than expected. Maybe another 30-40 minutes.

Jeremy Howard · 14 Nov 2002, 04:31 AM

OK, we've got the DB back, and we're just doing some testing now. We'll spend around 10-15 minutes testing if all goes well.

Fred · 14 Nov 2002, 04:38 AM

mmmh...

I'm on server 4 and can not login...

Is the server still down ?
Fred

Fred · 14 Nov 2002, 04:39 AM

oops, sorry, now it works.

kc80027 · 14 Nov 2002, 05:11 AM

Jeremy, thanks for the updates about the status of Server 4. I've noticed some discrepancies while trying to check on the status of Server 4 not through this forum. When I try to log on to my fastmail at https://www.fastmail.fm/mail4/login/ I get this message:

************************************************
FastMail.FM Outage

I'm sorry, FastMail.FM is currently down for 5 minutes for maintenance. Please check our status screen for updates, or use our forum to check for news or to request assistance.

Please check back in a few minutes to access your email.
************************************************

When I try to log on without specifying the server https://www.fastmail.fm/mail/login/ the message is:

************************************************
The server where your email is stored is currently down for maintenance. We expect it to be available in around 30 minutes.
************************************************

I know this seems nitpicking but you may want to be consistent with your downtime estimates. What is more bothersome is when I clicked on your link to check your status screen http://admin.fastem.com/cgi-bin/fm-admin.pl there was no indication of the server being down:

************************************************
The web server is UP, and responding, and logging in correctly. If you can't log in to it, you may have a problem with your account, or a problem with your connection to the Internet.
************************************************

When I checked the uptime history at http://admin.fastem.com/uptime.html there was no indication of Server 4 being down. The last entry for Server 4 "2002-11-13 12:54:07 Up" did not change for two hours.

The above checks were made while Server 4 was down. I just checked the uptime history and the next entry after 12:54:07 is "2002-11-13 15:00:06 Up" and again there is no indication that Server 4 was ever down.

Jeremy Howard · 14 Nov 2002, 05:17 AM

Thanks a lot for that feedback. Rob and I have high on our todo list an overhaul of our outage handling to remove some of these inconsistencies. Currently we don't have a very neat way of quickly reporting status through all systems.

14 Nov 2002, 02:57 AM	#1
Jeremy Howard Ultimate Contributor Join Date: Sep 2001 Location: Australia Posts: 11,501	server4 outage server4 has a database corruption problem. It will be down for a while--no ETA yet. No mail will be lost--it's being queued. More info as we have it...

14 Nov 2002, 03:10 AM	#3
DBH Moderator Join Date: Mar 2002 Location: USA Posts: 1,533	Open one of your messages and select Show full header next to the word Options in the top left corner. Look at the 2nd line that says 'Received:' ... somewhere in that line, you should see server5 . fastmail . fm (server5 is the server I'm on, at least) Hope this helps! Dave Last edited by DBH : 14 Nov 2002 at 03:14 AM.

14 Nov 2002, 03:05 AM	#2
chicory_patch Senior Member Join Date: Aug 2002 Location: NorthEast, USA Posts: 138	Jeremy, How can we find out which server we are on? Besides one day having no access because that server is down.

14 Nov 2002, 03:17 AM	#4
punchcutter Junior Member Join Date: Nov 2002 Posts: 7	Do I understand correctly from this exchange that each user is assigned permanently to a given server, rather than being assigned on a session-by-session (or connection-by-connection) basis?

14 Nov 2002, 03:27 AM	#5
Jeremy Howard Ultimate Contributor Join Date: Sep 2001 Location: Australia Posts: 11,501	Correct--your mail has to live somewhere. There are multiple servers for 'stateless' stuff however (e.g. SMTP).

14 Nov 2002, 03:28 AM	#6
m5comp Member Join Date: Nov 2002 Location: USA Posts: 54	Why can't server 4 users such as myself be switched temporarily to another server until the problem is fixed?

14 Nov 2002, 03:29 AM	#7
Jeremy Howard Ultimate Contributor Join Date: Sep 2001 Location: Australia Posts: 11,501	I'm guessing that this will take 30 minutes to resolve at this stage.

14 Nov 2002, 04:03 AM	#8
DarioMor Essential Contributor Join Date: Jun 2002 Location: Rio de Janeiro, Brasil Posts: 356	Only curiosity... but reading the full header makes me more confused. In wich server am I? Received: from www.fastmail.fm (server1.internal [10.202.2.132]) by server3.fastmail.fm (Cyrus v2.1.9) with LMTP; Tue, 12 Nov 2002 12:19:42 -0600 X-Sieve: CMU Sieve 2.2 Received: from server4.fastmail.fm (server4.internal [10.202.2.135]) by www.fastmail.fm (Cyrus v2.1.9) with LMTP; Tue, 12 Nov 2002 12:19:43 -0600 Received: from n13.grp.scd.yahoo.com (localhost [127.0.0.1]) by server4.fastmail.fm (Postfix) with SMTP id 39C242FD95

14 Nov 2002, 04:05 AM	#9
Jeremy Howard Ultimate Contributor Join Date: Sep 2001 Location: Australia Posts: 11,501	The 'by' of the first Received line is your server (server3).

14 Nov 2002, 04:06 AM	#10
Jeremy Howard Ultimate Contributor Join Date: Sep 2001 Location: Australia Posts: 11,501	My first guess was wrong, sorry. We're getting there, but the rebuild is taking longer than expected. Maybe another 30-40 minutes.

14 Nov 2002, 04:31 AM	#11
Jeremy Howard Ultimate Contributor Join Date: Sep 2001 Location: Australia Posts: 11,501	OK, we've got the DB back, and we're just doing some testing now. We'll spend around 10-15 minutes testing if all goes well.

14 Nov 2002, 04:38 AM	#12
Fred Junior Member Join Date: Oct 2002 Posts: 21	mmmh... I'm on server 4 and can not login... Is the server still down ? Fred

14 Nov 2002, 04:39 AM	#13
Fred Junior Member Join Date: Oct 2002 Posts: 21	oops, sorry, now it works.

14 Nov 2002, 05:11 AM	#14
kc80027 Member Join Date: Sep 2002 Location: Colorado Posts: 48	Jeremy, thanks for the updates about the status of Server 4. I've noticed some discrepancies while trying to check on the status of Server 4 not through this forum. When I try to log on to my fastmail at https://www.fastmail.fm/mail4/login/ I get this message: ********************************************** FastMail.FM Outage I'm sorry, FastMail.FM is currently down for 5 minutes for maintenance. Please check our status screen for updates, or use our forum to check for news or to request assistance. Please check back in a few minutes to access your email. ******************************************** When I try to log on without specifying the server https://www.fastmail.fm/mail/login/ the message is: ******************************************** The server where your email is stored is currently down for maintenance. We expect it to be available in around 30 minutes. ******************************************** I know this seems nitpicking but you may want to be consistent with your downtime estimates. What is more bothersome is when I clicked on your link to check your status screen http://admin.fastem.com/cgi-bin/fm-admin.pl there was no indication of the server being down: ******************************************** The web server is UP, and responding, and logging in correctly. If you can't log in to it, you may have a problem with your account, or a problem with your connection to the Internet. ********************************************** When I checked the uptime history at http://admin.fastem.com/uptime.html there was no indication of Server 4 being down. The last entry for Server 4 "2002-11-13 12:54:07 Up" did not change for two hours. The above checks were made while Server 4 was down. I just checked the uptime history and the next entry after 12:54:07 is "2002-11-13 15:00:06 Up" and again there is no indication that Server 4 was ever down.

14 Nov 2002, 05:17 AM	#15
Jeremy Howard Ultimate Contributor Join Date: Sep 2001 Location: Australia Posts: 11,501	Thanks a lot for that feedback. Rob and I have high on our todo list an overhaul of our outage handling to remove some of these inconsistencies. Currently we don't have a very neat way of quickly reporting status through all systems.