A Perfect Storm : Domain Change Made Harder Than It Has to Be

So in my last post, I noted that we upgraded to Sierra 2.4. Among other things, this is supposed to give us the _replace function for a wildcard, to reduce the security warnings users get when trying to go through the WAM to the databases.

Now, winter commencement was Dec. 15, 2016 (Thursday) but we had word that some instructors were allowing students to keep working until the 20th (Tuesday), so we asked for a delay to the upgrade to the 21st. That left the 22nd to check it out before we closed on the 23rd for the holiday break. Time zones were a little odd – it seems the Australia office was handling it. It went well.

Back to work on Jan. 3, 2017. Supportal didn’t like my login. Had to have that reset. Finally got in on Jan. 4 (Wednesday) and asked for the domain change. Got an email back from III: “I came to know from my support colleagues that you will be changing your domain and want to install the new SSL certificate immediate after that. Once your domain got changed and you are ready with the SSL, please raise the service commitment by visiting the URL and we will take care.”

Sounds good.

Friday 6th we were closed for snow, but I had a call in for our I.T. people already for the certificate. The catch apparently was, they were already tied up doing their own certificate updates just before the snow day. So they got back to me to generate a certificate on the 9th (Monday), and I tried to send that on the 10th. Supportal would not allow me to attach the certificate (which you would think would be allowed, but apparently was not). So, attached to the reply to the email. Later, converted to text and entered as comment in the Supportal.

I’m still getting no response on the domain change from III. No email either. I keep asking.

We have a holiday on Jan. 16 (Monday) and classes begin on the 17th. Got back and still no notice from III. I update Supportal and send email to the certificate person. Email auto-reply says he is out until the 19th.

The nice thing about banging your head against a brick wall is how much better it feels when you stop. I used the Supportal and email to postpone the domain change until spring break in March. Because if there is something we do not want to do, it’s try to change domains while classes are going on.<cue: foreboding music>

Finally got a response to contact them at the beginning of March to remind them. Will do.

March 8 Wednesday

And I thought I had a bad time in January

So, I had an email from the person on this back in January, to remind him at the start of March that we wanted to try this again March 20, the Monday of spring break week. That would leave us Tuesday, Wednesday and Thursday (we’re closed Friday) to make our changes at this end.

Provided it happens, of course.

I put it in the Supportal at the end of February. Thursday March 3rd I emailed. Monday the 6th I phoned (and had to leave voice mail for anyone who could help – this is a Service Desk?!?). Today I tried Supportal and called again, and at least confirmed he (let’s call him K) still works at III, but couldn’t answer the phone at the moment, and neither could the other person (let’s call him A) the Service Desk wanted to refer me to. Now I wait for the new person, A, to call back. At least I confirmed the first one still works there.

Since all I’m asking is to confirm a date still in the future, which was supposedly set in January, this is a disturbing lack of response.

And now Sierra is down. What the hey?!?!

Perfect Storm

Later Wednesday morning:

So, apparently III is having access problems with Syracuse on the network. So now I put in a call on that as well, since we can’t get the Sierra desktop app to connect.

Then I had a scheduled meeting to go to.

Meanwhile, my domain name change got referred to A, somebody new. Who apparently did NOT read the entire ticket, which said March 20th, and that I just wanted confirmation that it would happen on March 20th.

And who then went ahead, while I was out at a meeting, and changed our domain, set the _replace function, and installed the certificate.

Today. Immediately. With no preparation or coordination on our end, because we were planning on March 20.

That’s right – we are now libcat.uafs.edu but it doesn’t do us any good – almost EVERYTHING going through the server is broken. The WAM is useless, still no Sierra access, and — wait for it — the I.T. person on campus here who was going to help with this domain change IS OUT OF TOWN. I had to go over to the Service Desk in another building just to find that out.

Slow, deep breathing. Right.

I decided to not confuse III by trying to change back — they’re obviously busy enough as it is. We’ll try to rush this through with another one of our I.T. people. He’ll have to update the DNS (Domain Name System) so it recognizes the new domain name, and that has to propagate through all the others on the web. Plus anything else they need.

Then, IF we can finally get to our staff Sierra app, we can start testing to see what databases need changing in the WAM addresses (it seems some of them need hyphens added, but we have to have access to check).

Meanwhile, we have no circulation, we have to do a workaround to even use a limited number of databases from on campus… a perfect storm of impairments. At least the OPAC is still working online, for reasons I don’t know enough to understand. And I can still telnet in, for all the good that does me.

Yet later:

Finally our I.T. was able to get a good person on it, and he redirected our DNS to the new address. So:

  • In the Sierra app, update iiirunner.lax to the new address.
  • On the command line, enter ipconfig/flushdns to clear the DNS.

Start Sierra and now it works. The catalog is still working. BUT – the WAM is not connected to the databases, which was, after all, the primary reason for doing all this in the first place – so we could avoid the darn security warning pages that don’t like the iii.com SSL certificate.

Apparently our WAM is still putting the old address into the links. However, if you correct to the new address manually, you get:

This site can’t be reached

0-search.ebscohost.com.libcat.uafs.edu’s server DNS address could not be found.

So, update the Supportal and see what happens next.

Meanwhile, all our students studying for midterms are unhappy.

March 10 Friday

Looks like the libcat.uafs.edu domain is propagating through all the DNS servers on the web.

We got the off-campus databases working, but not the on-campus access. Our Ebsco Discover search through the WAM is still down, but we rigged a link to not go through the WAM and on-campus people can use that.

Then, we got attention from A on the matter, as well as the person (“B”) who originally recommended the domain change, and we got the wildcard put into our DNS which solved the on-campus access through the WAM: *.libcat.uafs.edu  We probably would have discussed that in advance of the change we expected to make on March 20.

Now, we have a change for the Discover search in the script on the home page, and that will be solved. We hope.

I’m still plowing through updating catalog links to ebooks. Past experience was that doing much more than, say, 50,000 a day frightens the server, so I’m doing it in chunks. Incidentally, we also had to update our 962 fields for linked images — that’s where, among other things, we put our donor/honoree bookplates.

So we seem to be coming out of a nosedive. I did a talk for the CAST college this morning, pre-changes, and this gets around the problem I had.

We may have to flushdns as I mentioned above, but we’re almost back.

TGIF!

March 15 Wednesday

Monday we seemed to be finally about back to fully operational. I got the last of the ebooks done Tuesday. And III has sent a couple of quality review requests for all this. I can hardly wait to see what I’m going to tell them….

Kudos to our I.T. person on dealing with the domain change in emergency mode!

As of Tuesday, I finished updating all the ebook records in the catalog (all 230,000 plus of them).

And today, I got the new director up on Sierra. Since Sierra installs with its own Java, there was a popup permission to give for that, which was hiding behind the Sierra logon screen, which is what was blocking it from going on.

And now we can look forward to training on our new ILL software from OCLC, which goes live here on March 31.

March is coming on like a lion, right – like Scar in the Lion King.

Scar in the Lion King