Comment by eduction

1 year ago

There wasn't any sign or promise in 2005 (that I'm aware of, and I had a key back then) that PGP would soon get meaningfully better, for example that it would suddenly, magically solve the issue of verifying keys in a decentralized system. It was a pain in the arse then as now. There has been some improvement in the meantime, in that keys.openpgp.org started unilaterally doing its own email verification.

And you say that it's been 18 years and PGP is behind best practices, but you don't describe how those best practices would better solve the package verification challenge that PyPi faces. So in the absence of an actual alternative system why not keep using PGP? Perfect is the enemy of good (IMO!).

But again, I'm not even really feeling strongly that PyPi should use PGP or not. I mostly posted just to say that they should be honest about why they are leaving it, and these seem like bad/misleading stats that for some reason they are hiding behind instead of coming out and saying they changed their mind about PGP (or new people are now running things and don't like dealing with PGP - many people would sympathize).

7 comments

eduction

Reply
woodruffw  1 year ago

> And you say that it's been 18 years and PGP is behind best practices, but you don't describe how those best practices would better solve the package verification challenge that PyPi faces. So in the absence of an actual alternative system why not keep using PGP? Perfect is the enemy of good (IMO!).

This may or may not be satisfying to you, but there is discussion around this, both in this thread, other threads on the internet, and PyPI's own issue tracker. The current plan is to integrate Sigstore[1] into PyPI as a more complete and modern codesigning solution. That work is progressing, and is not in a state that's meant to "replace" PGP. But that's intentional because, as the post states, nobody (to a first approximation) was using these PGP signatures anyways.

Perfect is indeed the enemy of the good; the other enemy of the good is bad things. PGP is bad; the reason I titled the original post "worse than useless" is because it takes a useless security feature (signatures that nobody verifies) and makes them actively dangerous by providing cryptographic margins that weren't even safe 25 years ago.

> But again, I'm not even really feeling strongly that PyPi should use PGP or not. I mostly posted just to say that they should be honest about why they are leaving it, and these seem like bad/misleading stats that for some reason they are hiding behind

Two things should be separated here: there's the PyPI blog post, which is written by the PyPI admins, and there's the "worse than useless" blog post, which was written by me. I am not an admin of PyPI, and it's my independent technical opinion that PGP is bad. I stand by the stats that I've included in my own post, but I do welcome specific critiques of how they're bad or misleading.

The PyPI admins can provide their own rationale, but this is my best understanding: they have known for years that PGP is bad, and have more or less tolerated it as a legacy feature because removing it was a low priority. The post I wrote two days ago was just a "final nudge" towards removing it, since the post's statistics (particularly large numbers of expired keys) refute one of the last defenses for PGP on PyPI.

[1]: https://www.sigstore.dev/

  • donaldstufft  1 year ago

    > The PyPI admins can provide their own rationale, but this is my best understanding: they have known for years that PGP is bad, and have more or less tolerated it as a legacy feature because removing it was a low priority. The post I wrote two days ago was just a "final nudge" towards removing it, since the post's statistics (particularly large numbers of expired keys) refute one of the last defenses for PGP on PyPI.

    PyPI Administrator here, and the person who removed GPG from PyPI.

    All the way back in 2013 I had written blog posts that talked about how GPG was not sufficient for a secure package signing scheme in a repository.

    I first proposed removing GPG back in May of 2016 (turns out May is a bad month for GPG in my world). At that time we were knee deep in rewriting PyPI into it's modern incanation and trying to quickly identify what features were actually important enough to keep in the new implementation and what features were not.

    Even back in 2016 I did not think that the level of use of GPG and the relative uselessness of the signatures made sense to keep it as a feature. However when I proposed it we got some small amount of push back, primarily from Linux distributors, and the feature had already been implemented so we just removed it from the UI and left the feature in. This wasn't an endorsement of the feature, but rather a tactical choice that it wasn't worth spending more time on removing GPG at that point when we were focused on the rewrite.

    In the intervening years it had periodically come up, everyone had agreed that it wasn't part of our long term plans, but nobody had the time to dig into figuring out if the signatures that were being uploaded were actually useful and without that, there was some vague concern that maybe somewhere out there some system might be relying on them, and not wanting to "pick a fight" over it at that time.

    Then woodruffw did the work to investigate how useful the existing signatures actually were, and quite frankly the numbers were worse than I expected. I honestly expected most of the existing signatures to be meaningfully verifiable, because from my perspective, the only people left signing were likely going to be people who were invested in GPG, and thus more likely to spend the time to make sure that everything was working.

    Given that new information, along with a long desire (over 7-10+ years now!) to remove this small bit of security theater, I went ahead and threw together a pull request to actually do it now. Like a lot of things in OSS, it was a perfect storm of someone pointing out a problem to someone who had enough time and motivation at that point in time to fix it that made that particular task bubble up to the top of my long TODO list.

    • eduction  1 year ago

      Why would you expect regular GPG users to have signed things years ago with keys that never expire? People who regularly use GPG tend to set expiration dates on their keys. Non expiring keys are a really bad idea.

      1 reply →