Building under Windows

[marshalc]

[Windows] 'cp' is not recognised - npm install issue

Hi folks,

We've just tripped over a small detail whilst introducing a new team member to our hugo+docsy build process. Whilst our CI processes are linux based, and so are are some devs (or Mac in my case), more of our team coming on board are Windows based, and they're having this problem.

We've traced the issue to a modification at 1966f9604c89875c7bfd83b66fbfee4128297ccf which was introduced a couple of months ago. Specifically package.json, line 11:

"_cp:bs-rfs": "cp -R node_modules/bootstrap/scss/vendor/* assets/_vendor/bootstrap/scss/",

Running an npm install on a Windows machine (from either PowerShell or CMD) results in an error of:

Now, I appreciate this is use using pre-release versions of the platform - but I wasn't sure if anyone else had found this yet.

The common answer to solving this is to monkey patch the code for Windows users to use copy or similar (and change the slashes in the command string too it seems); however this then breaks the code for the *nix users, so we don't like this option.

The next answer has been to get our Windows users to either install Windows Subsystem for Linux (WSL) and use bash there, or the git-bash that comes from with Git-for-windows as part of the mingw32 package.

What I'm wondering is if anyone else has tripped on this, and if so, have they come up with a multi-platform solution yet?

[deining]

What I'm wondering is if anyone else has tripped on this, and if so, have they come up with a multi-platform solution yet?

Yes a somehow similiar issue was rasied some time ago. In this context the cross-env package was identified as a potential multi-platform solution. In your case, I guess the shx package can be used to author a multi-platform solution. Can you try this package out, and potentially provide a pull request that solves your issue?

[marshalc]

They look like potential pathways to a cross platform answer, thanks - we can take a look at that and let you know what we find. If we can get it to work then we'll propose a PR fix.

[chalin]

Hi @marshalc. Just so I can better understand your use case: are you just trying to use Docsy as a dependency, or are you also attempting to built it yourselves?

[chalin]

@marshalc - Also, can you tell me how your are bringing in Docsy: NPM, Git submodule, or Hugo modules? Thx

[marshalc]

Thanks @deining - the shx solution appears to be working on the linux/Mac setups. I'm awaiting one of my windows-based colleagues testing it next week to confirm we're happy with it. The edit being tested is tiny (install shx, modify the one line in package.json to prepend shx to the cp line). You can see the edit at ouhft@f044c34

Hi @chalin - we're using Docsy in what I think is a typical theme dependancy use case in our various project's for documentation needs. Essentially it's project based docs, built with Hugo, themed by a lightly modified Docsy theme (see https://github.com/ouhft/docsy/tree/ouhft) added as a git sub-module (linked to that repo branch), and published via an internal Gitlab Pages build process to an internal only webserver (which unfortunately means showing build examples a tad more challenging though you can see the CI process in that link).

HTH

[chalin]

Thanks for the details @marshalc and the links to your project. Could you move shx into devDependencies instead?

My main problem is that I don't have access to a Windows environment so I can't validate the solutions I'd like to experiment with 🤷🏼‍♂️. (I'm looking forward to the likes of GitPod support for Windows :))

I finally started work on adding smoke tests (#1776), and I was able to validate (as I suspected) that Docsy can be used just fine under Windows. You're building Docsy, so that's another story.

I'd rather not add another pkg dependency if I can avoid it. I'm beginning to think that moving that prep step (involving the cp) into a bash script instead would be a simpler workaround. Would that work for you?

[chalin]

Ok, we now have smoke and built tests successfully passing under *nix and Windows. In particular you'll notice from this run https://github.com/google/docsy/actions/runs/7512943762/job/20454181915, that the cp command doesn't seem to cause a problem:

Can you give it a try @marshalc? If it doesn't work for you, I'm unsure what tooling GH might be using that is different from yours. Maybe WSL?

[marshalc]

Thanks for the updates - this response is a bit of a placeholder as in trying to investigate this compared to the path we were on has opened up a can of worms, and we're now in several rabbit holes (he says, badly mixing metaphors) trying to work out a clean approach that allows us to test your approach and then work out how it might apply in our context.

We did a quick test of our shx patch using one of our Windows desktops and that worked - but as you note, that adds quite a range of new package dependancies, so we'd like to be able to stay closer to the core approach and introduce as few divergencies for our branch as possible.

Our first challenge is we're using the git submodule rather than Hugo module approach and this seems to change the overall build and dependancies structures. We were unable to drop in the plain Docsy theme into any of our projects in place of our customised version (and are still trying to determine why), so testing the latest docsy:main hasn't worked - on either our Mac or Windows desktops.

To test some basic setup, we've been winding back to whether we can run the docsy:main user guide build in a clean download.. and we've had that work (but only if the local git repo folder is explicitly called docsy and not our usual docs-repo - so we're trying to find out why that specific path name appears to be a limit.

Our next step when we can return to this will be to try this first step on Windows desktop machines as well as my macOS one... then we work forward again, though I think that means we'll need to work out how to repackage our Docsy branch as a Hugo module first, before we can get to parity with you. (pointers welcome!)

One thing I noted from your output is that GH appears to be forcing the use of PowerShell in that setup, whereas npm on our local Windows desktop was running those sub-processes in cmd (even though it was called from a PowerShell prompt). The ins and outs of this are presently obscure to me.

[chalin]

Hi @marshalc, thanks for the updates.

Our first challenge is we're using the git submodule

All of my Hugo+Docsy-based projects in production are using Docsy via a git submodule. So, if you have specific questions about using Docsy in this way, let me know and I'll do my best to answer them.

To test some basic setup, we've been winding back to whether we can run the docsy:main user guide build in a clean download.. and we've had that work

Good.
Does that include Windows builds of the user guide?

(but only if the local git repo folder is explicitly called docsy and not our usual docs-repo - so we're trying to find out why that specific path name appears to be a limit.

You're hitting a Hugo requirement: if a Hugo project is configured to use a theme named abc, then Hugo will look for a folder with the same name under the themes directory. For details, see Theme components. So, if you put your Docsy theme in a folder named docs-repo, then it won't find it. You have two options: change the theme name to match the folder name or vice versa.

repackage our Docsy branch as a Hugo module first

That shouldn't be necessary -- we officially support use of Docsy as a git submodule.

forcing the use of PowerShell

Yes, that appears to be key.

---

Btw, since Docsy and the User Guide are successfully building under GH actions, I'm going to move this into the discussions area was we continue to explore your issues of building under Windows. Also note that I've tweaked the title.