FastCheck Forum Index FastCheck
Get notified when you receive new emails!
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

First fuzzy feelings for FastTest features

 
Post new topic   Reply to topic    FastCheck Forum Index -> FastTest
View previous topic :: View next topic  
Author Message
Jeremy Howard
The "e" in e-mail


Joined: 09 Apr 2002
Posts: 325
Location: Melbourne

PostPosted: Mon Apr 22, 2002 11:39 am    Post subject: First fuzzy feelings for FastTest features Reply with quote

So, the goal is to get all the diagnostics we need in one hit to find the problem source. V0.1 of FastTest simply needs to have a menu item for each test. V1.0 needs to run these in order and put a written report on the clipboard for sending. V2.0 needs to do some simple path analysis to only run the relevent tests and make suggestions at the problem source, and point to a FAQ entry with a potential solution. V3.0 would also examine installed software configurations and create an appropriate installation file (e.g. user.js for Mozilla, Outlook Express Settings File for Outlook Express, etc).

The ordering of what I write below is pretty arbitrary. V0.1 just implements the separate tests, so ordering isn't an issue.

First we provide a summary of the enviroment. MS Sysinfo under Windows, and some equivalent under Unix (which would be what?). ipconfig /all output.

Next, network connectivity test. Maybe we have a few servers in different places with different IPs, each running echo on every port. Try and find a machine we can connect to, then print out ports that we can reach (probably just try a couple of dozen standard ports).

Test ICMP connectivity--can we ping a well-known host? Assuming that we have ICMP connectivity, run a traceroute to www.fastmail.fm and admin.fastem.com. Can we ping fastmail.fm? rackspace.com?

Test the local machine to find what ports are open--find out if there's proxies on this machine, mail virus scanners, etc.

OK, now we check DNS. Can we resolve www.google.com or something common? www.dot.fm? www.fastmail.fm? www.fastem.com?

Can we ping the DNS servers set up for this machine? Can these servers identify the NS records for 'fm.'? Can it ping them? Can it identify the NS records for 'fastmail.fm'? Can it ping them? Is the IP address resolved for fastmail.fm correct?

Now let's check the web environment. Are we behind a forced proxy? --Test by telnetting to www.google.com port 80 and GET / HTTP/1.0. If there's a transparent proxy the lack of an HTTP/1.1 Host: header will cause it to fail. Are we using a configured proxy? We'll have to query the Windows environment to check (Linux doesn't really have a central place to select a proxy, does it?)

Can we get to http://admin.fastem.com/cgi-bin/fm-admin.pl? According to this, is FastMail.FM currently UP? (We should create a different version of fm-admin.pl that returns simple XML status, so it's easier to parse).

Can we get to http://www.fastmail.fm? Can we get to http://www.fastmail.fm/mail/?FLN-UserName=$User&FLN-Password=$Pwd ? Does it return "Welcome to FastMail" like we expect? (Obviously we'll need an edit box to enter a username and password). Did the user enter a sensible username and password? No caps or '@domain' in the username? For web testing, we'll need to use their proxy configuration--Eric, we'll need some help from you in understanding how to handle this...

Now, lets do some port connection checks to fastmail.fm and *.proxy.fastmail.fm. Can we connect to the port (IMAP/POP3/SMTP/HTTP)? Does it return the banner we expect? How about the *.proxy.fastmail.fm ports (using an open port identified in the echo test earlier).

Then, let's connect to a "status CGI" that returns information about that user's service level, whether their account is locked, and so forth. The CGI doesn't exist yet, but we'll use the same authentication system as FastCheck and just create a little CGI to return the user data in a basic XML structure (again, like FastCheck).

Once we've found open ports for each of the four services, test logging into each using one of the ports we've found for each service. Use the status CGI results to only test those ports accessible for this users service level.

Well, that was a pretty jumbled description of everything. Sorry... I've only recently started thinking about this. Anyhoo, at least you know how far I've got with this design to date, so please post your ideas or questions below. Smile
Back to top
View user's profile Send private message Visit poster's website
gpdoyon



Joined: 12 Apr 2002
Posts: 8
Location: Westbrook, Maine

PostPosted: Mon Apr 22, 2002 2:07 pm    Post subject: Wow! Reply with quote

Okay, that's enough for a start! Very Happy

I may be able to begin working on it during the week here at work when things are slow. I've not got too much time during the nights but have plenty of time usually on the weekends.

I'll look in to some of the example code from the Indy modules and see what I can start with.
Back to top
View user's profile Send private message
eric_thiebaut
Site Admin


Joined: 07 Apr 2002
Posts: 1524
Location: Sydney, Australia

PostPosted: Mon Apr 22, 2002 6:48 pm    Post subject: Reply with quote

I think jeremy summed up quite well what we want to do.

Just to add a couple comments: most people who will use this will either be fastmail beginners, or internet beginners (usually, experienced users either won't have any problems, or will know how to detect & fix them, or wait if something is down).
My point is: KISS. The app should be extremely easy to use and user interaction should be minimal (probably limited to asking them their user name & password).
We should try to find out ourselves everything we can. Like the proxy settings for ex. There is the standard IE location in the registry, but I doubt all flavours of all browsers (Opera, Mozilla and the like) all use the same location? I could easilly install half a dozen browsers and figure out where they store their info. I can do the same on Unix too. It's probably in a .whatever file in the user's home directory.

Also, the sequence of tests should be automated as much as possible. I reckon V0.1 should just iterate through all tests and run them all. V0.2 or 1.0 could be clever and have a flowchart of 'what to do in case this works or not'.

I think our next steps could be:
* Define a public interface for a unit test. A unit test can either succeed or fail, it can read some parameters (username/password) from somewhere, and displays its output to a provided stream (standard output...)
* Define the list of unit tests we want to do (Jeremy's lists looks quite good)
* Write the series of unit tests as a Delphi library
* Write a wrapper application which runs all tests (V0.1)
* I could also include the same library in FastCheck
* I can find out where all browsers store their proxy settings.
* Regarding the 'downloading a URL [through a firewall]' bit, my code is actually Microsoft C++ ATL (minus a couple bug fixes Wink ). We can either use a Delphi library (I'm sure this exists) or create one, or I could also create a Delphi-friendly library on top of the ATL classes. That wouldn't take me long.

Eric
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Jeremy Howard
The "e" in e-mail


Joined: 09 Apr 2002
Posts: 325
Location: Melbourne

PostPosted: Tue Apr 23, 2002 7:17 am    Post subject: Reply with quote

eric_thiebaut wrote:
We should try to find out ourselves everything we can. Like the proxy settings for ex. There is the standard IE location in the registry, but I doubt all flavours of all browsers (Opera, Mozilla and the like) all use the same location? I could easilly install half a dozen browsers and figure out where they store their info. I can do the same on Unix too. It's probably in a .whatever file in the user's home directory.
Perhaps someone who is a non-programmer could take on this job? There's plenty of programming jobs to keep us busy...
eric_thiebaut wrote:
* Define a public interface for a unit test. A unit test can either succeed or fail, it can read some parameters (username/password) from somewhere, and displays its output to a provided stream (standard output...)
I'll try and do this soon. Eric, if you find a good URL downloading solution for Windows let me know.
Back to top
View user's profile Send private message Visit poster's website
mklose



Joined: 12 Apr 2002
Posts: 84
Location: Germany

PostPosted: Wed Apr 24, 2002 12:30 am    Post subject: Reply with quote

OK, some comments.

I agree this should be a KISS application. Click a button and it if it can reach fastmail directly, directly send in the log to fastmail to a certain account, assign a unique ID, put this in the subject line and open a dialog, where the user can enter some details, and then can choose to send the data or ignore.
If anyone knows Mozilla's talkback agent, this is what I am thinking about.

Then display the unique ID which can be used as a reference in an e-mail to you should the user have problems. Also display all past send IDs. Many people will try fastcheck and not all will have problems. So store them somewhere, but don't specifically monitor them.
Of course, if sending doesn't work, copy all the stuff in the clipboard or save it to a text file, or both.

Maybe we could have an additional option to send the webmasters a mail, which refers to this "talkback" ID.

This is the GUI part.

In future versions, apart from talkback, we could have some sort of AI (I am thinking about a tree structure which asks questions until you reach a leaf which would be the diagnoses of the problem or a link to the FAQ - IBM has a nice implementation of this on their website for Thinkpad notebooks).

Now to the behind the scenes work:

Tools we need to implent or find libraries:

- ping
- traceroute (both trivial to implement yourself in 5 minutes)
- our own nslookup


We should start from the top:

We should resolve the hostname of fastmail.org (using the normal library function) check it against the iP address of fastmail.fm and then open a port 80 TCP connection to it, and get the current status as Jeremy discussed. If that works, display a message:

"DNS OK"
"Connection with fastmail possible"
"Fastmail is up"

If all this is OK, then let us do a simple POP and then an IMAP test (I wouldn't use the user's account, I would set up a test account exactly for this purpose) and try logging in with USER and PASS and maybe do a LIST request. in the case of POP a similar thing for iMAP)

If that works: "POP Server UP"
"IMAP server up"

If this works, then let us try the SSL ports.

now to try SMTP. Let us just login, do a HELO but I wouldn't try sending a mail at this stage.

If all that is working, end the test. If access to the fastmail site was possible, but the website states the server is down: display message that it is down and end test.


OK,now let's assume that we weren't able to contact fastmail.

We have a problem.

We need to discuss what problems can occur.

First thing to check:

Do we have an internet connection? (I'd start with a ping, but some firewalls block ICMP packet and TCP connections is what we are really interested in).
OK, so what do we do? I would suggest as a firststep to open a port 80 connection to google.com and just see if the webserver responds. I would do this in 2 steps:

1) try and resolve the hostname, if it doesn't work, we have a serious DNS problem. I would end the test here and start gathering stats. Without a working DNS, nothing works (except web proxies)
2) actually open the TCP connection and say hello to the webserver. If this doesn't work, then stop and gather information.

Trying to access google.com over port 80 will also tell us whether we are behind a transparent proxy or not.

OK, google works. TCP works, at least over port 80.

We already know if the DNS lookup to fastmail worked or not.

If the DNS lookup succeeded, but we can't get to it, then we need to do a traceroute. Traceroute. If ICMP is blocked this will not work, but there is not much else we can do at this stage. We know we have an internet connection, it works to google over port 80, but not to fastmail, eventhough the IP address is correct.

If the initial DNS lookup for fastmail did not succeed (but did for www.google.com) then we have something weird going on like the guy yesterday where his ISP was rerouting all traffic on port 53 going to IP addresses not belonging to them to the "standard" upon dialup assigned DNS.
What I would do here is do here is query ns1.zoneedit.com for fastmail.fm, check that the data is indeed coming from ns1.zoneedit.com and not some other nameserver.

Now we need to gather stats.

Windows:

I would run the shell commands:
ipconfig /all
read the host file
run route print

registry:

log the run/run once keys (this is where viruses and resident programs write themselves to)
And a lot of other stuff too. Most important, the exact version of windows.

Linux

run /sbin/ifconfig
read /etc/resolv.conf
read /etc/hosts
run /sbin/route -n

just some ideas.
Back to top
View user's profile Send private message AIM Address
ady



Joined: 12 Apr 2002
Posts: 175
Location: Indonesia

PostPosted: Wed Apr 24, 2002 12:55 am    Post subject: Reply with quote

Jeremy Howard wrote:
Perhaps someone who is a non-programmer could take on this job? There's plenty of programming jobs to keep us busy...


Let me help. I don't have programming experience under Windows. I used to do Pascal, Foxpro, C and assembly under DOS time.

But I can play around the Windows registry. Sometimes it's a fun place to play Laughing
Back to top
View user's profile Send private message
gpdoyon



Joined: 12 Apr 2002
Posts: 8
Location: Westbrook, Maine

PostPosted: Wed Apr 24, 2002 10:17 am    Post subject: Reply with quote

Jeremy Howard wrote:
Eric, if you find a good URL downloading solution for Windows let me know.


We can pobably stage the code/executables on my home ISP's web site. I'll check to see what the bandwidth allowance is. It should last us for a while, or, at least until it needs to get downloaded by many people.

- Gerry
Back to top
View user's profile Send private message
eric_thiebaut
Site Admin


Joined: 07 Apr 2002
Posts: 1524
Location: Sydney, Australia

PostPosted: Wed Apr 24, 2002 10:50 am    Post subject: Reply with quote

I think Jeremy meant 'a library or some source that enables us to download something from a URL', like the stuff I use for FastCheck.

I'll try to find one, or convert the A++ code I use in FastCheck.

Eric
Back to top
View user's profile Send private message Send e-mail Visit poster's website
gpdoyon



Joined: 12 Apr 2002
Posts: 8
Location: Westbrook, Maine

PostPosted: Wed Apr 24, 2002 11:42 am    Post subject: Reply with quote

I see Eric. Thanks!
Back to top
View user's profile Send private message
eric_thiebaut
Site Admin


Joined: 07 Apr 2002
Posts: 1524
Location: Sydney, Australia

PostPosted: Wed Apr 24, 2002 11:45 am    Post subject: Reply with quote

Oops, just realised I wrote A++. I swear, I didn't use that (whatever that is!) for Fastcheck Embarassed

Eric
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    FastCheck Forum Index -> FastTest All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group