Running into build problems trying to build for an IBM Power9 system

[austinpagan]

Before I dig too deep into this, I just wanted to know whether vmd-python was ever intended to run on Power systems...? If so, I will append the particular problem I'm seeing.

[Eigenstate]

Oh boy... yes regular VMD will run on anything, including Power8. When making vmd-python I removed a lot of support for architectures I didn't feel comfortable supporting. Post your error and we can totally get it working!

[austinpagan]

First problem I ran into is that in /usr/include/gnu, the stubs.h file was trying to include stubs-32.h, but that file was not present. I installed everything I could think of, and couldn't get that fixed. So I just hacked stubs.h to point to the file which WAS present (stubs-64-v2.h).

Then, when I tried to do "python setup.py build" again, I got this:

[root@234bc51125a5 vmd-python]# python setup.py build
running build
running build_py
package init file 'vmd/init.py' not found (or not a regular file)
package init file 'vmd/init.py' not found (or not a regular file)
Compiling VMD
Finding libraries...
LIB: libtcl8.5.so -> /lib64
INC: tcl.h -> /usr/include
WARNING: Could not find library file 'libsqlite3.so' in standard library directories.
Defaulting to: '/usr/lib/libsqlite3.so'
LIB: libsqlite3.so -> /usr/lib

WARNING: Could not find include file 'sqlite3.h' in standard include directories.
Defaulting to: '/usr/include/sqlite3.h'
INC: sqlite3.h -> /usr/include/sqlite3.h
LIB: libexpat.so -> /lib64
INC: expat.h -> /usr/include
WARNING: Could not find library file 'libnetcdf.so' in standard library directories.
Defaulting to: '/usr/lib/libnetcdf.so'
LIB: libnetcdf.so -> /usr/lib

WARNING: Could not find include file 'netcdf.h' in standard include directories.
Defaulting to: '/usr/include/netcdf.h'
INC: netcdf.h -> /usr/include/netcdf.h
Building with:
CC: gcc
CXX: g++
CFLAGS: -I/usr/include
CXXFLAGS: -I/usr/include
LDFLAGS: -L/usr/lib64 -Wl,--no-as-needed,-rpath-link,/usr/lib64
Setting environment variables
Deleting previous plugin directory /vmd-1.9.3/vmd-python/build/lib/vmd/plugins
Compiling plugins to /vmd-1.9.3/vmd-python/build/lib/vmd/plugins
Building Molecule File Reader plugins
/usr/bin/ld: cannot find crti.o: No such file or directory
/usr/bin/ld: skipping incompatible /usr/lib/gcc/ppc64le-redhat-linux/4.8.5/libstdc++.so when searching for -lstdc++
/usr/bin/ld: cannot find -lstdc++
/usr/bin/ld: skipping incompatible /usr/lib/gcc/ppc64le-redhat-linux/4.8.5/libgcc.a when searching for -lgcc
/usr/bin/ld: cannot find -lgcc
/usr/bin/ld: skipping incompatible /usr/lib/gcc/ppc64le-redhat-linux/4.8.5/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: cannot find -lgcc_s
/usr/bin/ld: skipping incompatible /usr/lib64/libc.so when searching for -lc
/usr/bin/ld: skipping incompatible /usr/lib64/libc.a when searching for -lc
/usr/bin/ld: skipping incompatible //usr/lib64/libc.so when searching for -lc
/usr/bin/ld: skipping incompatible //usr/lib64/libc.a when searching for -lc
/usr/bin/ld: cannot find -lc
/usr/bin/ld: skipping incompatible /usr/lib/gcc/ppc64le-redhat-linux/4.8.5/libgcc.a when searching for -lgcc
/usr/bin/ld: cannot find -lgcc
/usr/bin/ld: skipping incompatible /usr/lib/gcc/ppc64le-redhat-linux/4.8.5/libgcc_s.so when searching for -lgcc_s
/usr/bin/ld: cannot find -lgcc_s
/usr/bin/ld: cannot find crtn.o: No such file or directory
collect2: error: ld returned 1 exit status
make[2]: *** [abinitplugin.so] Error 1
make[1]: *** [molfilelibs] Error 1
make: *** [LINUX] Error 2
Traceback (most recent call last):
File "setup.py", line 377, in
'test': VMDTest,
File "/usr/lib64/python2.7/distutils/core.py", line 152, in setup
dist.run_commands()
File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "setup.py", line 39, in run
self.execute(self.compile_vmd, [], msg="Compiling VMD")
File "/usr/lib64/python2.7/distutils/cmd.py", line 349, in execute
util.execute(func, args, msg, dry_run=self.dry_run)
File "/usr/lib64/python2.7/distutils/util.py", line 324, in execute
func(*args)
File "setup.py", line 59, in compile_vmd
check_call(cmd, cwd=srcdir)
File "/usr/lib64/python2.7/subprocess.py", line 542, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/vmd-1.9.3/vmd-python/vmd/install.sh', 'LINUX', '/vmd-1.9.3/vmd-python/build/lib/vmd', '/usr']' returned non-zero exit status 2
[root@234bc51125a5 vmd-python]#

[Eigenstate]

I'll add a check for Power9 systems in setup.py and put the right logic back in the configure script which hopefully will fix your problems with plugins not building.
What's the output of:
python -c "import platform; print(platfom.system(), platform.machine()"?

[austinpagan]

You had some typos, but here's what I think you want:

$ python -c "import platform; print(platform.system(), platform.machine())"
('Linux', 'ppc64le')
$

[austinpagan]

Just FYI, I would be very happy to act as an alpha test site, as you work to get this running on Power architectures. I will not be at all disappointed if this becomes an iterative process over several days.

[austinpagan]

Also, a question: I see that your implementation supports offscreen buffers (what I have come to know as "openglpbuffer"). My belief is that a prerequisite for painting to such pbuffers is the use of EGL in VMD, but I'm not sure that is true. In any case, does your VMD-as-a-Python-library have the option to use EGL?

[Eigenstate]

Alright, I've made a new branch, ppc64le with an initial attempt. The build logic for this architecture seems to be pretty much identical except for no linkage to libstdc++ (hence the errors you were seeing earlier). Let me know if it works!

I am in the middle of polishing a new release and that does include support for offscreen rendering with EGL if you build with python setup.py build --egl but this is pretty experimental for now. You'll need the EGL development headers and (EGL/egl.h) and the OpenGL entry point library (libOpenGL.so) for the build to succeed.

[austinpagan]

So, I did these things:

git clone --branch ppc64le https://github.com/Eigenstate/vmd-python
pip install --upgrade pip
pip install --trusted-host pypi.python.orig numpy
cd vmd-python
python setup.py build

The last command generated this output:

[vmd-python]# python setup.py build
running build
running build_py
package init file 'vmd/init.py' not found (or not a regular file)
package init file 'vmd/init.py' not found (or not a regular file)
Compiling VMD
Finding libraries...
LIB: libtcl8.5.so -> /usr/lib64/
INC: tcl.h -> /usr/include/
error: None
[vmd-python]#

I will now attempt the "python setup.py install" and the next comment will contain those results.

[austinpagan]

heh.

"python setup.py install" gave me almost the exact same output as "python setup.py build":

[vmd-python]# python setup.py install
running install
running build
running build_py
package init file 'vmd/init.py' not found (or not a regular file)
package init file 'vmd/init.py' not found (or not a regular file)
Compiling VMD
Finding libraries...
LIB: libtcl8.5.so -> /usr/lib64/
INC: tcl.h -> /usr/include/
error: None
[vmd-python]#

Maybe that missing init.py file is actually something important?

[austinpagan]

For future reference, can you point me to the EGL/egl.h and the libOpenGL.so?

[Eigenstate]

Hmmmm, that's weird. It's not the init.py, that's a benign error as that file is moved from elsewhere during installation.
Although the output is missing it probably is failing to find libsqlite3. I've committed some better error messages. What distro are you running?

On Debian, you'll need the following packages for the dependencies:

• libegl1-mesa-dev
• libopengl0
• libsqlite3-dev
• libnetcdf-dev
• libexpat1-dev
• tcl8.5-dev

[austinpagan]

Here's what I see when I type "cat /etc/os-release":

NAME="CentOS Linux"
VERSION="7 (AltArch)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (AltArch)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7:server"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

[austinpagan]

So, I tried on a simpler P9 system, with fewer bells and whistles, and it did give me more debug info:

[fossum@fstgbfen1 ~]$ git clone --branch ppc64le https://github.com/Eigenstate/vmd-python
Cloning into 'vmd-python'...
remote: Enumerating objects: 46, done.
remote: Counting objects: 100% (46/46), done.
remote: Compressing objects: 100% (32/32), done.
remote: Total 9038 (delta 25), reused 30 (delta 14), pack-reused 8992
Receiving objects: 100% (9038/9038), 39.11 MiB | 14.21 MiB/s, done.
Resolving deltas: 100% (3733/3733), done.
Checking out files: 100% (2498/2498), done.
[fossum@fstgbfen1 ~]$ cd vmd-python
[fossum@fstgbfen1 vmd-python]$ python setup.py build
running build
running build_py
package init file 'vmd/init.py' not found (or not a regular file)
package init file 'vmd/init.py' not found (or not a regular file)
Compiling VMD
Finding libraries...
LIB: libtcl8.5.so -> /usr/lib64/
INC: tcl.h -> /usr/include/
Traceback (most recent call last):
File "setup.py", line 407, in
'test': VMDTest,
File "/usr/lib64/python2.7/distutils/core.py", line 152, in setup
dist.run_commands()
File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "setup.py", line 66, in run
self.execute(self.compile_vmd, [], msg="Compiling VMD")
File "/usr/lib64/python2.7/distutils/cmd.py", line 349, in execute
util.execute(func, args, msg, dry_run=self.dry_run)
File "/usr/lib64/python2.7/distutils/util.py", line 324, in execute
func(*args)
File "setup.py", line 77, in compile_vmd
self.set_environment_variables(pydir)
File "setup.py", line 284, in set_environment_variables
os.environ["SQLITELIB"] = "-L%s" % self._find_library_dir("libsqlite3")
File "setup.py", line 201, in _find_library_dir
libfile);
RuntimeError: Could not find library 'libsqlite3.so' in standard library directories. Update $LD_LIBRARY_PATH to include the directory containing this file, or make sure it is present on your system
[fossum@fstgbfen1 vmd-python]$

On this system, when I type "cat /etc/os-release" I get this:

[fossum@fstgbfen1 vmd-python]$ cat /etc/os-release
NAME="Red Hat Enterprise Linux Server"
VERSION="7.5 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.5"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.5 (Maipo)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.5:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.5
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.5"
[fossum@fstgbfen1 vmd-python]$

[austinpagan]

I searched for libsqlite3.so in my /usr directory, and it is not there. Should I be doing a 'yum install' of one or more of the following:

sqlite-devel.ppc64le : Development tools for the sqlite3 embeddable SQL database
sqlite-doc.noarch : Documentation for sqlite
sqlite-tcl.ppc64le : Tcl module for the sqlite3 embeddable SQL database engine
sqlite2-devel.ppc64le : Development files for SQLite
sqlite2-tcl.ppc64le : Tcl bindings for sqlite
sqlite3-dbf.ppc64le : Converter of XBase / FoxPro tables to SQLite
sqlite.ppc64le : Library that implements an embeddable SQL database engine
sqlite2.ppc64le : Embeddable SQL engine in a C library

?