15 Nov 2013 robbat2   » (Master)

python-exec: solutions for package conflicts, and making it easier on users

Running into another system today with the fun python-exec block, I realise that while it has been discussed on the Gentoo mailing lists, and the forums slightly, there's been hardly any posts about it in the blog stream.

I'm not going to go into what caused it, but rather solutions for package conflicts in the short term, and also the long-term. The TL;DR general solution is running "emerge -1 dev-python/python-exec"

Here's the latest conflict I got on it; I wanted to install mirrorselect to compare some hosts

hostname / # emerge -pv mirrorselect

These are the packages that would be merged, in order:
[ebuild  N     ] net-analyzer/netselect-0.3-r3  22 kB
[ebuild     U  ] dev-lang/python-2.7.5-r3:2.7 [2.7.3-r2:2.7] USE="gdbm hardened%* ipv6 ncurses readline ssl threads (wide-unicode) xml -berkdb -build -doc -examples -sqlite -tk -wininst" 10,026 kB
[ebuild     U  ] dev-lang/python-3.2.5-r3:3.2 [3.2.3:3.2] USE="gdbm hardened%* ipv6 ncurses readline ssl threads (wide-unicode) xml -build -doc -examples -sqlite -tk -wininst" 9,020 kB
[ebuild  N     ] dev-lang/python-exec-2.0:2  PYTHON_TARGETS="(jython2_5) (jython2_7) (python2_6) (python2_7) (python3_2) (-pypy2_0) (-python3_3)" 79 kB
[ebuild  N     ] dev-util/dialog-1.2.20121230  USE="nls unicode -examples -minimal -static-libs" 422 kB
[ebuild  N     ] app-portage/mirrorselect-2.2.0.1  PYTHON_TARGETS="python2_7 python3_2 -python2_6 (-python3_3)" 13 kB
[blocks B      ] <dev-python/python-exec-10000 ("<dev-python/python-exec-10000" is blocking dev-lang/python-exec-2.0)

Total: 6 packages (2 upgrades, 4 new), Size of downloads: 19,580 kB
Conflict: 1 block (1 unsatisfied)

 * Error: The above package list contains packages which cannot be
 * installed at the same time on the same system.

  (dev-python/python-exec-0.2::gentoo, installed) pulled in by
    dev-python/python-exec[python_targets_python2_7(-),-python_single_target_python2_5(-),-python_single_target_python2_6(-),-python_single_target_python2_7(-)] required by (dev-libs/libxml2-2.9.0-r2::gentoo, installed)

  (dev-lang/python-exec-2.0::gentoo, ebuild scheduled for merge) pulled in by
    dev-lang/python-exec:=[python_targets_python2_6(-)?,python_targets_python2_7(-)?,python_targets_python3_2(-)?,-python_single_target_python2_6(-),-python_single_target_python2_7(-),-python_single_target_python3_2(-)] (dev-lang/python-exec:=[python_targets_python2_7(-),python_targets_python3_2(-),-python_single_target_python2_6(-),-python_single_target_python2_7(-),-python_single_target_python3_2(-)]) required by (dev-python/setuptools-0.6.30-r1::gentoo, installed)
    dev-lang/python-exec:=[python_targets_python2_6(-)?,python_targets_python2_7(-)?,python_targets_python3_2(-)?,python_targets_python3_3(-)?,-python_single_target_python2_6(-),-python_single_target_python2_7(-),-python_single_target_python3_2(-),-python_single_target_python3_3(-)] (dev-lang/python-exec:=[python_targets_python2_7(-),python_targets_python3_2(-),-python_single_target_python2_6(-),-python_single_target_python2_7(-),-python_single_target_python3_2(-),-python_single_target_python3_3(-)]) required by (app-portage/mirrorselect-2.2.0.1::gentoo, ebuild scheduled for merge)
    dev-lang/python-exec:=[python_targets_python2_6(-)?,python_targets_python2_7(-)?,python_targets_python3_2(-)?,python_targets_python3_3(-)?,python_targets_pypy2_0(-)?,-python_single_target_python2_6(-),-python_single_target_python2_7(-),-python_single_target_python3_2(-),-python_single_target_python3_3(-),-python_single_target_pypy2_0(-)] (dev-lang/python-exec:=[python_targets_python2_7(-),python_targets_python3_2(-),-python_single_target_python2_6(-),-python_single_target_python2_7(-),-python_single_target_python3_2(-),-python_single_target_python3_3(-),-python_single_target_pypy2_0(-)]) required by (virtual/python-argparse-1::gentoo, installed)

For more information about Blocked Packages, please refer to the following
section of the Gentoo Linux x86 Handbook (architecture is irrelevant):

http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?full=1#blocked

This system has just dev-lang/python-exec-2.0 presently. We can reduce the conflict down to a minimal version as follows: 

HOST / # emerge -pv  dev-lang/python-exec

These are the packages that would be merged, in order:
[ebuild  N     ] dev-lang/python-exec-2.0:2  PYTHON_TARGETS="(jython2_5) (jython2_7) (python2_6) (python2_7) (python3_2) (-pypy2_0) (-python3_3)" 79 kB
[blocks B      ] <dev-python/python-exec-10000 ("<dev-python/python-exec-10000" is blocking dev-lang/python-exec-2.0)

Total: 1 package (1 new), Size of downloads: 79 kB
Conflict: 1 block (1 unsatisfied)

 * Error: The above package list contains packages which cannot be
 * installed at the same time on the same system.

  (dev-python/python-exec-0.2::gentoo, installed) pulled in by
    dev-python/python-exec[python_targets_python2_7(-),-python_single_target_python2_5(-),-python_single_target_python2_6(-),-python_single_target_python2_7(-)] required by (dev-libs/libxml2-2.9.0-r2::gentoo, installed)

  (dev-lang/python-exec-2.0::gentoo, ebuild scheduled for merge) pulled in by
    dev-lang/python-exec
    dev-lang/python-exec:=[python_targets_python2_6(-)?,python_targets_python2_7(-)?,python_targets_python3_2(-)?,-python_single_target_python2_6(-),-python_single_target_python2_7(-),-python_single_target_python3_2(-)] (dev-lang/python-exec:=[python_targets_python2_7(-),python_targets_python3_2(-),-python_single_target_python2_6(-),-python_single_target_python2_7(-),-python_single_target_python3_2(-)]) required by (dev-python/setuptools-0.6.30-r1::gentoo, installed)
    dev-lang/python-exec:=[python_targets_python2_6(-)?,python_targets_python2_7(-)?,python_targets_python3_2(-)?,python_targets_python3_3(-)?,python_targets_pypy2_0(-)?,-python_single_target_python2_6(-),-python_single_target_python2_7(-),-python_single_target_python3_2(-),-python_single_target_python3_3(-),-python_single_target_pypy2_0(-)] (dev-lang/python-exec:=[python_targets_python2_7(-),python_targets_python3_2(-),-python_single_target_python2_6(-),-python_single_target_python2_7(-),-python_single_target_python3_2(-),-python_single_target_python3_3(-),-python_single_target_pypy2_0(-)]) required by (virtual/python-argparse-1::gentoo, installed)

For more information about Blocked Packages, please refer to the following
section of the Gentoo Linux x86 Handbook (architecture is irrelevant):

http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?full=1#blocked

So what do we know?

  1. We have dev-python/python-exec-0.2 installed, it has the default SLOT=0.
  2. Here's what the packages in the tree right now look like: 
    $ egrep '^R?DEPEND|^SLOT' dev-{python,lang}/python-exec/*ebuild
dev-python/python-exec/python-exec-10000.1.ebuild:SLOT="0"
dev-python/python-exec/python-exec-10000.1.ebuild:RDEPEND="dev-lang/python-exec:0[${PYTHON_USEDEP}]"
dev-python/python-exec/python-exec-10000.2.ebuild:SLOT="2"
dev-python/python-exec/python-exec-10000.2.ebuild:RDEPEND="dev-lang/python-exec:0[${PYTHON_USEDEP}]
dev-lang/python-exec/python-exec-0.3.1.ebuild:SLOT="0"
dev-lang/python-exec/python-exec-0.3.1.ebuild:RDEPEND="!<dev-python/python-exec-10000"
dev-lang/python-exec/python-exec-0.9999.ebuild:SLOT="0"
dev-lang/python-exec/python-exec-0.9999.ebuild:RDEPEND="!<dev-python/python-exec-10000"
dev-lang/python-exec/python-exec-2.0.ebuild:SLOT="2"
dev-lang/python-exec/python-exec-2.0.ebuild:RDEPEND="!<dev-python/python-exec-10000"
dev-lang/python-exec/python-exec-2.9999.ebuild:SLOT="2"
dev-lang/python-exec/python-exec-2.9999.ebuild:RDEPEND="!<dev-python/python-exec-10000"
  3. If we try to bring in dev-lang/python-exec directly, it will trigger the block, because our version of dev-python/python-exec is too old.
  4. This entire problem happens because the python*r1 eclasses bring in dev-lang/python-exec.

This leads to a simple user-actionable solution of "emerge -1 dev-python/python-exec", which will work as follows (notice that portage uninstalls the old version for us):

HOST / # emerge -pv  dev-python/python-exec
These are the packages that would be merged, in order:
[ebuild  N     ] dev-lang/python-exec-0.3.1  PYTHON_TARGETS="(jython2_5) (jython2_7) (python2_6) (python2_7) (python3_2) (-pypy2_0) (-python3_3)" 73 kB
[ebuild  N     ] dev-lang/python-exec-2.0:2  PYTHON_TARGETS="(jython2_5) (jython2_7) (python2_6) (python2_7) (python3_2) (-pypy2_0) (-python3_3)" 79 kB
[uninstall     ] dev-python/python-exec-0.2  PYTHON_TARGETS="(jython2_5) (jython2_7) python2_5 (python2_6) (python2_7) python3_1 (python3_2) -pypy1_9 (-pypy2_0) (-python3_3)" 
[blocks b      ] <dev-python/python-exec-10000 ("<dev-python/python-exec-10000" is blocking dev-lang/python-exec-2.0, dev-lang/python-exec-0.3.1)
[ebuild  NS    ] dev-python/python-exec-10000.2:2 [0.2:0] PYTHON_TARGETS="(jython2_5) (jython2_7) (python2_6) (python2_7) (python3_2) (-pypy2_0) (-python3_3)" 0 kB

Total: 3 packages (2 new, 1 in new slot, 1 uninstall), Size of downloads: 152 kB
Conflict: 1 block

The above is not actually the minimal solution, but it is the best general solution. The minimal solution is to include the slot on the package, but in future if the slots change further and the default slot is removed, this won't work anymore.

HOST / # emerge -pv dev-python/python-exec:0
These are the packages that would be merged, in order:
[ebuild  N     ] dev-lang/python-exec-0.3.1  PYTHON_TARGETS="(jython2_5) (jython2_7) (python2_6) (python2_7) (python3_2) (-pypy2_0) (-python3_3)" 73 kB
[ebuild     U  ] dev-python/python-exec-10000.1 [0.2] PYTHON_TARGETS="(jython2_5) (jython2_7) (python2_6) (python2_7) (python3_2) (-pypy2_0) (-python3_3) (-pypy1_9%) (-python2_5%*) (-python3_1%*)" 0 kB
[blocks b      ] <dev-python/python-exec-10000 ("<dev-python/python-exec-10000" is blocking dev-lang/python-exec-0.3.1)

Total: 2 packages (1 upgrade, 1 new), Size of downloads: 73 kB
Conflict: 1 block

But now the better question, is as developers, can we help users prevent this, and at what cost? If we don't mind new users having an extra placeholder package, then yes, we CAN actually solve it for the users. In all of the dev-lang/python-exec ebuilds we need to make this simple change:

 RDEPEND="!<dev-python/python-exec-10000"
+PDEPEND=">=dev-python/python-exec-10000:$SLOT"

This provides a nice solution as follows:

# emerge -pv dev-lang/python-exec
These are the packages that would be merged, in order:
[ebuild  N     ] dev-lang/python-exec-0.3.1  PYTHON_TARGETS="(jython2_5) (jython2_7) (python2_6) (python2_7) (python3_2) (-pypy2_0) (-python3_3)" 73 kB
[ebuild     U  ] dev-python/python-exec-10000.1 [0.2] PYTHON_TARGETS="(jython2_5) (jython2_7) (python2_6) (python2_7) (python3_2) (-pypy2_0) (-python3_3) (-pypy1_9%) (-python2_5%*) (-python3_1%*)" 0 kB
[blocks b      ] <dev-python/python-exec-10000 ("<dev-python/python-exec-10000" is blocking dev-lang/python-exec-2.0, dev-lang/python-exec-0.3.1)
[ebuild  N     ] dev-lang/python-exec-2.0:2  PYTHON_TARGETS="(jython2_5) (jython2_7) (python2_6) (python2_7) (python3_2) (-pypy2_0) (-python3_3)" 79 kB
[ebuild  NS    ] dev-python/python-exec-10000.2:2 [0.2:0] PYTHON_TARGETS="(jython2_5) (jython2_7) (python2_6) (python2_7) (python3_2) (-pypy2_0) (-python3_3)" 0 kB

Total: 4 packages (1 upgrade, 2 new, 1 in new slot), Size of downloads: 152 kB
Conflict: 1 block

All that remains is convincing the Python team to accept this solution for users...

Syndicated 2013-11-15 21:34:32 from Move along, nothing to read

Latest blog entries     Older blog entries

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!