First attempt at migrating to Drupal 9 -> many questions

Hi there,

I’ve learnt through your mail of October 18th that all web sites will be migrated to openshift and drupal 9 and that we “need to CHECK the preview of your website in OpenShift before 2 November”.
I’ve started to try to follow the “Step by step guide” but before I ask questions about it, I have a more general one : when is the dead line for the migration ? Porting non trivial sites with custom modules will be (again !) quite some amount of work, and I have no clue when we manage to find time for this. It will take months for sure.
Now concerning the guide itself, I have failed miserably on pretty much every steps, so here are my questions :

  • when checking the website, first tab is the remove tab. And you said “We thus encourage experienced users to do this by mounting their website’s WebDAV”. I’ve tried but there is absolutely no trace of the listed modules in the WebDAV mount. Where are they supposed to be ?
  • second tab is the update one. I got one module to update. It’s now done (and broke the site :frowning: but that’s not the point). How do I “refresh” the list to check it’s now ok ?
  • I then have my two custom modules split into one “collaborate with maintainers” and one “Fix manually”. Is there any difference between the 2 ?
  • finally for the custom modules, you mention “follow guidance documentation that will be added to https://drupal-tools.web.cern.ch/ in the coming days”. Any news on this ? Is it somewhere ?

Thanks,

Sebastien

PS : the site I’ve tested is https://runningclub.webtest.cern.ch/

1 Like

Adding one more question here : it seems that the links are not properly translated from “.web.cern.ch” to “.webtest.cern.ch” on the test site. This breaks a number of things, e.g. our sponsor images on the right panel which come in tiny size (16x16) with title “This image has been removed. For security reasons, only images from the local domain are allowed.”

Is there any way around this, so that we can find out the “real” problems form the ones induced by this “feature”.

Hi Sebastien

Thanks for reaching out!

Please rest assured that your site(s) will not be shut down if you have not completed the upgrade by the 2nd of November. However, as this is the day on which Drupal 8 reaches end-of-life, websites still running Drupal 8 will be exposed to fast growing security risk. As such, if sites remain on Drupal 8 much longer after this date, the security risk necessitates them being made publicly unavailable (but accessible on the CERN network) until administrators have ensured compatibility.

We have gone ahead and updated the guide based on feedback, please check again:

https://drupal-tools.web.cern.ch/how-to-prepare-your-website-its-drupal-9-upgrade

Hopefully it is a bit clearer!

In regards to your questions:

  1. Custom modules can be found under ModulesCustom. If you do not see any modules here, your website does not have any custom modules installed and is unlikely to need much input beyond perhaps an update. What is displayed in your Upgrade Status report?
  2. I assume you mean refreshing the Upgrade Status report? If yes, simply tick the module(s) in question and conduct a new scan to confirm any changes or updates you have completed. This will refresh the page and you should get a new status on the module(s) in question.
  3. Which modules do you see under COLLABORATE WITH MAINTAINERS? If you see Webforms, all is well. If you see other modules, please investigate whether you actually require these modules. A module will be listed under this section if there is no apparent work being done to ensure Drupal 9 compatibility (e.g. if the maintainers are inactive, or if the module simply stopped being supported). The FIX MANUALLY is where you need to actively do something, e.g. pasting a core_version_requirement line as described in the above guide.
  4. The first articles have gone live today. We are finalising these as fast as possible.

In regards to your newest question concerning the .webtest.cern.ch and .web.cern.ch: This is of course unfortunate — particularly when you are supposed to verify that things look the way they are supposed to — but please rest assured that once your actual site is migrated to OpenShift (scheduled for this week), the domain will remain unchanged, thus no longer triggering this particular issue.

Thanks!

Thanks a lot for you detailed answer, and for updating the guide !

Concerning the site, it’s ok to keep it on drupal 8 for some time with visibility only within CERN. As you can guess, most of our members are at CERN anyway. Just a question : how shall I trigger the migration when we are ready ?

Regarding the other questions :
1- There is no such entry “Modules → Custom” in my site, neither under drupal 8, nor on the test site under drupal 9. My own custom module (rcc_module) is installed directly in “Modules”. So I’m still not really sure what you mean by “it is generally good practice to properly remove modules” for the ones in the remove list. And I have 33 there !
2- yes, that’s what I meant. I’ll do as you suggest although I’m pretty sure that’s what I’ve done already and it did not change anything. I’ve retried right now, but I get " 504 Gateway Time-out"… I suppose the service is in bad shape right now
3- under “Collaborate with maintainers”, I see my own custom module (rcc_module/RunningClub). I’ll go and collaborate with myself to fix it :slight_smile: Under “Fix manually”, I see my custom theme (CERN Override) which was fixing a couple of defects of the main CERN theme for my web site. There also, I’ll fix. I probably can drop it if the defects are gone. So indeed, not much difference between the 2 sections ?
5- .webtest.cern.ch vs .web.cern.ch: I’m puzzled by your answer here : will the migration occur this week or not ? You were saying we would stay on drupal 8. Or do you mean that we will migrate to openshift, but still under Drupal 8 ? If yes, where can we test this ?

Sebastien

Ok, I’m progressing a tiny bit : I’ve sorted out the CERN Override theme (simple rebase was enough) and updated it.

I think I understand also why the rescan does not change things : because I assumed this was restarting from my original web site and thus I was updating package versions there. I now suppose that I should update package version on the webtest site and redo it if fine on the original site. Is that correct ? Can you maybe make this clear in the documentation if it it correct ?

Now I’m still unable to check anything tonight, server seems to not respond for scans… But I have anyway another issue : trying to update Calendar package on the webtest site, the update complains that “You are not using an encrypted connection, so your password will be sent in plain text.” and then says “To continue, provide your server connection details”. Is that another “feature” of the webtest setup ? Did I do something wrong ?

Hi Sebastien

I need to accentuate that while you naturally have time to make your website Drupal 9 compatible, we strongly encourage everyone to get it resolved as fast as possible (hence us stressing November 2nd!). We very much appreciate the short notice, and understand that it can be hard to allocate resources in this manner. However, we hope that the Upgrade Status tool will help ease the process; even in instances where work is necessary. As for triggering the migration when you are ready: We will be at your disposal for that, please just let us know!

I see. This differs slightly from the norm. In general, all custom modules will be present in the Modules → Custom subfolder. This way, you can easily distinguish between modules which you actually have installed yourself, and modules which were included as part of your initial site creation.

All modules which show up in the REMOVE section are modules which are not actually used by your website. Instead, they are either modules which have been centrally distributed (e.g. a bunch of CERN ... modules), or they are modules which your website at one point or another has utilised. The actual list is going to be revised slightly as we try and shorten it centrally (e.g. remove some of the CERN ... modules from showing).

The reason why we recommend users to examine the modules listed here despite them being safe to ignore is that every module will still show up under the Extend tab. This means that if incompatible modules are left lingering around, there is a risk someone inadvertently attempts to install what would then be an incompatible module months into the future. This could break the website in question.

In your specific case, no, not so much!

Apologies for any confusion!

Allow me to rephrase: Two things are happening these weeks. The first is all websites being moved to the OpenShift infrastructure. This is a separate operation from the Drupal 9 upgrade entirely, though it happens around the same time! The move to OpenShift itself eases a lot of other tasks, including those concerning managing Drupal across CERN.

The .webtest.cern.ch website is a preview of your production website on the OpenShift infrastructure. It is there to both give you a chance to inspect how things look as well as peak at the Upgrade Status report, thereby allowing you to understand what, if anything, needs updating on your production website(s) prior to upgrading to Drupal 9. The .webtest.cern.ch version is on a Drupal 8.9 installation. It is not yet upgraded to Drupal 9.

Happy to hear this, thanks for the update!

Correct. While the .webtest.cern.ch is a preview version, you can think of it is a test website in that regard. Changes applied to the .webtest.cern.ch website are not going to persist on your production website; though at the same time, any potentially breaking changes are not going to affect your production website either. I will make this clearer in the documentation; thanks for the feedback!

This is not something we have had reported before.

We will publish revised Update and Removal guides later today… hopefully these will resolve this.

Thanks for your many detailed responses and questions!

Thanks a lot for your detailed answer. I’m slowly progressing, so let me summarize where I am :

  • no worries about getting things resolved. I’m on it. Just that depending what it takes, it can easily last several weeks. But see below, I have good hope it won’t :slight_smile:
  • I still do not get where the “Removed” modules should be removed. I see none of them in webdav, so I’ll suppose that it’s the report that is wrong and lists things which are not there. And note that it’s far from being only CERN* ones. E.g. easychart, WebForm or file entity
  • got it for webtest being still drupal 8. So where/how can I test drupal 9 version of my site once I’m done with the updating ?
  • still not able to update Calendar on webtest due to the “using an encrypted connection” problem
  • concerning my main package (rcc_module), I’ve updated it the core_version_requirement to say it can deal with drupal 9 and it has disappeared from the scan page :slight_smile: I suppose it means that it has a high chance to work on drupal 9, so maybe I’m already close ?

Cheers,

Sebastien

No worries! You are likely seeing a lot of modules listed under the REMOVE section which you never actually used. There are a number of reasons for why this is, though since you do not have any of these modules visible in WebDAV, you can safely dismiss this section. Once on the new infrastructure, we will be able to trim the list people see in the REMOVE section as we can manage modules centrally, thus removing a lot of confusion.

You should have received an e-mail informing you when your website is scheduled for migration to OpenShift. I suggest submitting a ticket requesting a test site to be created alongside the migration for you to test your Drupal 9 changes. You can do that here: https://cern.service-now.com/service-portal?id=sc_cat_item&name=Drupal-Clone&se=Drupal-Service.

This is possibly something we need to follow up with you on. At the moment, I suggest retrying once your production site has been migrated to the new infrastructure. If problems persist, it would probably be helpful to create a separate thread for this matter.

Indeed! If it has passed a new scan, you should be all good!

Great ! Seems that next step for me is sit and relax until the openshift migration takes place in 2 days :slight_smile: Thanks again for the help.

Yes!

And of course, should something unexpected crop up, please do not hesitate letting us know!

Thanks!

Hi,

back here after I’ve done many unsuccessful attempts to solve my problem. Short summary :

  • the openshift migration broke the site, in particular page https://runningclub.web.cern.ch/plan-entrainement-trail-printemps-2021
  • I’ve opened a ticket but got no solution
  • we thus decided to undo the migration. But site is still broken anyway !
  • the problem is that I need to install an external library (sibyx/phpgpx) but it seems impossible in current site as you can only install it with composer and composer was not supported
  • now composer starts to be supported, but I need to create a new site to be able to use it. I’ve followed the instructions at Creating a drupal site with extra configuration [Temporary] but my new site does not seem to be created…

So I’m stuck again, and getting bored of so many things failing one after the other and lack of support via tickets. Thanks by the way for the support given here, it’s way more valuable than the answers I go t to my tickets.

Are you able to tell me why site test-rcc-drupal9 in openshift does not get created ? Alternatively, do you know how to install an external library on an existing site without composer ?

Thanks,

Sebastien

Hi Sebastien

Thanks for raising this.

I appreciate how continuously having things fail can be frustrating, and I do apologise for you not receiving any response on your ticket(s). I do not immediately see any tickets from you on my end, though there are multiple teams working on this. If you have shared additional information in your ticket, please shoot me an e-mail with the ticket ID and I will have a look.

In any event, I will have a look at this tomorrow and get back to you ASAP.

We appreciate your continued patience.

Thanks!