Re: [integration-dev] global-jjb vs. packer vs. Jenkins jobs

Anil Belur

Greetings Robert:

We'll need to pin the cryptography module < 3.4 since rust dependencies are broken upstream pyca repo.

This issue should be addressed once this is merged. 


On Mon, Feb 8, 2021 at 6:58 AM Robert Varga <nite@...> wrote:
On 05/02/2021 10:16, Robert Varga wrote:
> On 03/02/2021 00:03, Anil Belur wrote:
>> Greetings Robert:
> Hello Anil,

Hello again,

sorry, for self-reply, but as it happens ...


>> 2. All releng/global-jjb (templates) scripts do not require all of the
>> PyPi dependencies to be installed and are tied down to the $job or
>> $project, since this approach binding them all into the same env has a
>> risk of the deps being broken more frequently.
>> 3. PyPi libs/modules are updated more frequently.
> While that is true, this line of reasoning completely ignores the
> failure mode and recovery.
> As it stands any of:
> - busted global-jjb
> - PyPi package updates
> - PyPi repository unavailability
> As we have seen in these past weeks, any such failure immediately
> propagates to all jobs and breaks them -- resulting in nothing working
> anymore, with no real avenue for recovery without help of LF IT.

... we just got hit by this.


> I am sorry, but I fail to see how Python packages special enough to inflict:
> - breakages occurring at completely random times

A case in point:
just failed with:

> [yangtools-maven-verify-master-mvn35-openjdk11] $ /bin/bash /tmp/
> --->
> Generating Requirements File
>   ERROR: Command errored out with exit status 1:
>    command: /usr/bin/python3 /home/jenkins/.local/lib/python3.6/site-packages/pip/_vendor/pep517/ build_wheel /tmp/tmpyskb3tqv
>        cwd: /tmp/pip-install-ibprnn2v/cryptography_c4b114d9e7a14d488215cb74c3a04315
>   Complete output (144 lines):


>   writing manifest file 'src/cryptography.egg-info/SOURCES.txt'
>   running build_ext
>   generating cffi module 'build/temp.linux-x86_64-3.6/_padding.c'
>   creating build/temp.linux-x86_64-3.6
>   generating cffi module 'build/temp.linux-x86_64-3.6/_openssl.c'
>   running build_rust
>       =============================DEBUG ASSISTANCE=============================
>       If you are seeing a compilation error please try the following steps to
>       successfully install cryptography:
>       1) Upgrade to the latest pip and try again. This will fix errors for most
>          users. See:
>       2) Read for specific
>          instructions for your platform.
>       3) Check our frequently asked questions for more information:
>       4) Ensure you have a recent Rust toolchain installed.
>       =============================DEBUG ASSISTANCE=============================
>   error: Can not find Rust compiler
>   ----------------------------------------
>   ERROR: Failed building wheel for cryptography
> ERROR: Could not build wheels for cryptography which use PEP 517 and cannot be installed directly
> Build step 'Execute shell' marked build as failure comes from global-jjb. This means jobs are
currently broken because of global-jjb stopped working.

I have filed
and we are blocked on it. Let's see what sort KPIs that issue will have.


Join { to automatically receive all group messages.