Page MenuHome GnuPG

gpg --multifile and wildcards problem on Windows
Closed, WontfixPublic

Description

There are reports that gpg used with wildcards on Windows does not work
correctly since we switched tim mingw64. We need to check wthether there are
differences in the C startup code of both's toolchains.

Details

Version
2.2.0

Event Timeline

Hi,
there is apparently a compile option "enable-wildcard" for the mingw-w64 c runtime.

But from:
http://mingw-w64.sourcearchive.com/documentation/0~20100125-3/wildcard_8c-source.html

I take it that you can also enable it in the program by defining int _dowildcard

-1;

I'll test it and let you know if this worked.

Mmmh when I run:

#include <stdio.h>
int _dowildcard = -1;
int main (int argc, char **argv)
{

int i = 0;
for ( i = 0; i< argc; i++) {
    printf("%s\n", argv[i]);
}
return 0;

}

With the command line *.* it just prints "*.*". Linking with CRT_glob.o directly
also leads to the same result.

I've tested this again with the mingw version that is part of ubuntu and it works.

I've commited a patch to gpg4win to enable this and will check if this also
works with the debian wheezy mingw-crt version.

aheinecke added a project: Restricted Project.

Ok using the same compiler I've used a year ago I also get the same result I got
a year ago (phew I thought i screwed up the test).

But with the patch this should just work once we change the used mingw-version.
I could not find a bugreport or some other information where and when this is fixed.

aheinecke removed a project: Restricted Project.Aug 13 2014, 12:35 PM

Now for 2.2.2 we've switched to a new dedicated buildsystem with a fully updated
debian wheezy whereas the old buildsystem was a debian unstable snapshot with
the same mingw Version: 2.0.3-1
I could not find an upstream bug or any indication that there was a fix for this.

But with beta-37 of 2.2.2 I was able to use wildcards on the command line.
I am out of my wits why this works now.

Emanuel: can you confirm that wildcards now work on the command line so that we
can mention this in the news and maybe look at integrating the patch into gnupg?
I'd like a different pair of eyes on that behavior. Maybe I've just messed up my
testsystem.

aheinecke added a subscriber: emanuel.

Tested this myself again. And I feel comfortable that this is fixed. Still I do
not know why this works now.

Werner can you take a look at the patch I've added for this:

http://git.gnupg.org/cgi-bin/gitweb.cgi?p=gpg4win.git;a=blob;f=patches/gnupg2-2.0.26/0001-Enable-wildcard-expansion-with-mingw-w64.patch;h=6c5b9f8b0794cf32ab909c0a340f81377dc3d42f;hb=HEAD

Is this something you would consider including in some form in gnupg or should I
move the patch into the generic gnupg patch directory?

It would be rather surprising if just one tool does globbing on Windows but all
the otehrs don't do it. It may actually lead to bugs due to broken quoting in
mingw. I recall than many years ago we had a lot of problems with the ever
changing quoting in the globbing implementation.

In any case using globbing with --multifile is not a good idea at all. There is
always a limit on this and thus the proper way of handling this is to use xargs
or whatever the counterpart is on Windows.

I disagree, all builtin windows tools do globbing so you expect it on the
command line. And as this is default for software that is built with visual
studio and with some versions of mingw it is more likely to see a software that
supports globbing then not.

With regards to quoting problems. I think that users would prefer some problems
instead of just "no". There is no xargs equivalent in Windows by default and so
you have to fall back to really ugly batch scripts that do this in for loops.

Yes one could argue that those users should use a "real shell" on a "real
system" ;-)

Well, it's part of 2.2.2 now, lets see if we get some feedback on this and maybe
for the next release we can revisit the question again if we should enable this
or not.

It is not useful for that purpose. This is probably the reason why we received
no complaints about it unless someone uses --multifile with filenames. It is
almost always wrong to do this.

Not sure what you mean by "It is not useful for that purpose" what do you mean
by "It" and "that" in this context.

With regard to complaints there are reports about this that have been active
nearly all the time since this was broken by 2.2.0:
https://wald.intevation.org/forum/message.php?msg_id=2569&group_id=11

This thread triggered your initial opening of this issue.

I agree that wildcard / multifile it is not very important. But I think we
should fix it as it is a small annoyance and it worked with gpg 2.1.0. That it
stopped working was due to our switch to mingw-w64 and not due to a decision
made by us to disable this. So I would see this as a Bug.

Some more threads about this:
https://wald.intevation.org/forum/forum.php?thread_id=1119&forum_id=21&group_id=11
https://wald.intevation.org/forum/message.php?msg_id=3404

P.S.
Yes I dislike the message boards too but they are surprisingly active.

To use --multifile with files on the command line. Such a use will suddenly
break. This is the same as using grep on a large number of files - only rookies
do that.

Now, shall I add this to gnupg 2.1? To which tools? All or just gpg?

Well just gpg would be enough imho as this is by far the most prominent command
line tool.

On the other hand it might be more prudent for us to hack / patch it just in the
gpg4win build to have it enabled globally for all tools we ship so that it is
more consistent. This would mean patching the compiler tough which we tried to
avoid so far.

I would be fine with moving this patch to the version independet gnupg2 patches
in gpg4win as it is kind of a "distribution" option forced upon gpg4win by the
compiler we are currently using.

Werner: If you agree please give a short ping here and I'll move the patch /
close the issue.

I won't do that just for gpg - this would be inconsistent. The wrapper we put
into the PATH directory needs this as well. What about gtk and qt libraries -
they run exe files internally - will the quoting continue to work? A single '*'
in a file name would likely break Enigmail.

Windows does not allow file names with a '*'. I'm not sure on what level but Its
ok not to handle this case.
I don't expect any problems for internal usage. Keep in mind that this is a
regression, we had wildcard expansion before we made the switch to mingw-w64.

We also don't need this in gpgwrap as gpgwrap just passes the argument on and it
will be expanded in the process itself.

But I actually like the idea to do the wildcard expansion in kleowrap / gpgwrap.
This way it would be contained in Gpg4win and we catch all our "user exposed"
processes. Ok?

For gpg4win 3.0 this will be a problem again as I no longer patch it.

We have to find a solution here. I do not find it acceptable that gnupg does not
understand globs on windows.

aheinecke removed a project: Restricted Project.Aug 28 2015, 11:06 PM

I just commited the fix I had in gpg4win 2e71bf3. I don't see how this is objectionable as it changes the behavior back to what we had before we switched to building on jessie and is a minor ifdef.

aheinecke changed the task status from Resolved to Wontfix.Apr 25 2017, 10:11 AM

Talked to Werner about this. He still has concerns that this is wrong because an application should not do globbing itself and only changing this in gpg.exe is inconsistent. It also might be a security problem as most users won't use the double dash to seperate arguments from filenames.

I'm pretty sure that we will get Bug reports about this as for users it will feel like a regression and we got reports quickly the last time we lost globbing (after switching to mingw-w32) but we should document Bash / Powershell snippets to show how the use case (mainly decrypting multiple files in a scheduled job afaik) can still be done.

Macro okboss:
It's not a bug it's a feature.