编译GreenPlum源码时遇见的问题及解决方式

greenplum

发布日期: 2022-03-08

文章字数: 2k

阅读时长: 10 分

阅读次数:

前段时间编译了 greenplum 源码，过程中遇到了一些问题，在这里进行一下记录，希望能够帮助遇见相同问题的小伙伴更快的解决问题。

Could not fetch URL https://pypi.python.org/simple/conan/:

错误信息

Could not fetch URL https://pypi.python.org/simple/conan/: There was a problem confirming the ssl certificate: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:765) - skipping

解决方式

可以执行如下命令：

# 解决 1 
pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org tools
# tools 为你要安装的工具
pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org --upgrade pip

# 如果以上都不解决问题，可是使用 -i 参数指定国内的镜像来进行安装

pip install -i https://pypi.tuna.tsinghua.edu.cn/simple psutil

如果没有pip，在上面解决问题时会报如下错误：

Could not fetch URL https://pypi.python.org/simple/pip/: There was a problem confirming the ssl certificate: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:765) - skipping
  Could not find a version that satisfies the requirement pip (from versions: )
No matching distribution found for pip

解决方式

# 1. 使用 yum 安装 pip
sudo yum -y install epel-release
sudo yum -y install python-pip

# 2. 使用 python 安装 pip
python get-pip.py --trusted-host pypi.org --trusted-host

library ‘xslt’ is required for XSLT support

错误信息

checking for xsltCleanupGlobals in -lxslt... no
configure: error: library 'xslt' is required for XSLT support

解决方式

yum install libxslt libxslt-devel

could not read symbols: Invalid operation

/bin/ld: libpq/SUBSYS.o: undefined reference to symbol 'gss_delete_sec_context@@gssapi_krb5_2_MIT'
/bin/ld: note: 'gss_delete_sec_context@@gssapi_krb5_2_MIT' is defined in DSO /lib64/libgssapi_krb5.so.2 so try adding it to the linker command line
/lib64/libgssapi_krb5.so.2: could not read symbols: Invalid operation
collect2: error: ld returned 1 exit status
make[2]: *** [postgres] Error 1
make[2]: Leaving directory `/home/greenplum/src/backend'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/home/greenplum/src'
make: *** [all] Error 2

解决方式

由于我们使用的是CentOS系统, 所以一定要加上/usr/local/lib和/usr/local/lib64的路径到/etc/ld.so.conf文件中，这一步尤为重要，目的是将常用的动态函数库加载到内存中，可以提高函数库的访问效率。另外，如果不配置这一步，在后面编译的过程中会遇到一些奇奇怪怪的缺少引用的错误

修改后的文件内容类似如下：

cat /etc/ld.so.conf
include ld.so.conf.d/*.conf
/usr/local/lib
/usr/local/lib64

# 修改完成后，运行 ldconfig 命令

could not determine flags for linking embedded Perl

错误信息

checking for flags to link embedded Perl... Can't locate ExtUtils/Embed.pm in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .).
BEGIN failed--compilation aborted.
no
configure: error: could not determine flags for linking embedded Perl.
This probably means that ExtUtils::Embed or ExtUtils::MakeMaker is not installed.

解决办法

yum install perl-ExtUtils-Embed -y

library ‘xml2’ (version >= 2.6.23) is required for XML support

错误信息

checking for xmlSaveToBuffer in -lxml2... no
configure: error: library 'xml2' (version >= 2.6.23) is required for XML support

解决方式

yum install libxml2 libxml2-devel -y

Your ORCA version is expected to be 2.51.XXX

错误信息

checking Checking ORCA version... configure: error: Your ORCA version is expected to be 2.51.XXX

解决方式

这说明你装的orca的版本是不匹配的，需要安装匹配版本，那么如何在安装前知道需要什么版本，

# 执行如下命令：
cd /home/gpdb_src/depends
cat conanfile_orca.txt

#这里在文件中可以看到如下记录：
[requires]
orca/v2.51.0@gpdb/stable

到github上找到对应版本下载安装即可。

当需要更换版本时，需要清理掉之前已经安装的版本。默认情况下安装在/usr/local/目录下，根据安装的路径

# 执行如下命令删除已经安装的版本
rm -rf build/*
rm -rf /usr/local/include/naucrates
rm -rf /usr/local/include/gpdbcost
rm -rf /usr/local/include/gpopt
rm -rf /usr/local/include/gpos
rm -rf /usr/local/lib/libnaucrates.so*
rm -rf /usr/local/lib/libgpdbcost.so*
rm -rf /usr/local/lib/libgpopt.so*
rm -rf /usr/local/lib/libgpos.so*

注意：

1、build位于gporca编译目录，其是编译时cmake产生的一些文件

2、如果重新安装了版本一定要执行ldconfig命令，否则新的版本不会在缓存中更新，会一直报上述错误

command ‘gcc’ failed with exit status 1

错误信息

unable to execute gcc: No such file or directory
    error: command 'gcc' failed with exit status 1

    ----------------------------------------
Command "/usr/bin/python2 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-4lu7Y0/psutil/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-YPgXFP-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-4lu7Y0/psutil/

解决方式

yum install -y gcc

could not find function ‘gss_init_sec_context’ required for GSSAPI

错误信息

checking for library containing gss_init_sec_context... no
configure: error: could not find function 'gss_init_sec_context' required for GSSAPI
# 如果使用编译参数：--with-gssapi可能会报该错误信息

解决方式

# 安装gssapi
pip install gssapi

安装 gssapi 可能报如下错误：

Complete output from command python setup.py egg_info:
    In distributed package, building from C files...
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-CHeKis/gssapi/setup.py", line 98, in <module>
        raise Exception("Could not find main GSSAPI shared library.  Please "
    Exception: Could not find main GSSAPI shared library.  Please try setting GSSAPI_MAIN_LIB yourself or setting ENABLE_SUPPORT_DETECTION to 'false'

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-CHeKis/gssapi/

解决方式

yum install -y krb5-devel.x86_64

apr-1-config is required for gpfdist, unable to find binary

错误信息

checking for apr-1-config... no
configure: error: apr-1-config is required for gpfdist, unable to find binary

解决方式

yum install -y apr-devel

libevent is required for gpfdist

错误信息

checking for library containing event_add... no
configure: error: libevent is required for gpfdist

解决方式

yum install -y libevent-devel

Library requirements (curl) not met.

错误信息

checking for curl-config... no
*** The curl-config script could not be found. Make sure it is
*** in your path, and that curl is properly installed.
*** Or see http://curl.haxx.se/
configure: error: Library requirements (curl) not met.

解决方式

yum install -y libcurl-devel.x86_64

header file <bzlib.h> is required for bzip2 support

错误信息

checking for bzlib.h... no
configure: error: header file <bzlib.h> is required for bzip2 support

解决方式

yum install -y bzip2-devel

psutil/_psutil_common.c:9:20: fatal error: Python.h: No such file or directory

安装psutil时可能报如下错误：

psutil/_psutil_common.c:9:20: fatal error: Python.h: No such file or directory
     #include <Python.h>
                        ^
    compilation terminated.
    error: command 'gcc' failed with exit status 1

    ----------------------------------------
Command "/usr/bin/python2 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-3gzCd8/psutil/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-EAY9ck-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-3gzCd8/psutil/

解决方式

yum install -y python-devel.x86_64

Error 256 while executing cd ‘/home/gpdb_src/gpd . . .

错误信息

ERROR: orca/v3.21.0@gpdb/stable: Error in build() method, line 48
        cmake.configure(defs=cmake_defines)
        ConanException: Error 256 while executing cd '/home/gpdb_src/gpdb-5.16.0/depends/.conan/data/orca/v3.21.0/gpdb/stable/build/4f7d6d5032b1a188f98e0c149ef6bf91e76af63e' && cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE="Release" -DCONAN_EXPORTED="1" -DCONAN_IN_LOCAL_CACHE="ON" -DCONAN_COMPILER="gcc" -DCONAN_COMPILER_VERSION="4.8" -DCONAN_CXX_FLAGS="-m64" -DCONAN_SHARED_LINKER_FLAGS="-m64" -DCONAN_C_FLAGS="-m64" -DCONAN_LIBCXX="libstdc++" -DBUILD_SHARED_LIBS="ON" -DCMAKE_INSTALL_PREFIX="/home/gpdb_src/gpdb-5.16.0/depends/.conan/data/orca/v3.21.0/gpdb/stable/package/4f7d6d5032b1a188f98e0c149ef6bf91e76af63e" -DCMAKE_INSTALL_BINDIR="bin" -DCMAKE_INSTALL_SBINDIR="bin" -DCMAKE_INSTALL_LIBEXECDIR="bin" -DCMAKE_INSTALL_LIBDIR="lib" -DCMAKE_INSTALL_INCLUDEDIR="include" -DCMAKE_INSTALL_OLDINCLUDEDIR="include" -DCMAKE_INSTALL_DATAROOTDIR="share" -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY="ON" -Wno-dev '/home/gpdb_src/gpdb-5.16.0/depends/.conan/data/orca/v3.21.0/gpdb/stable/build/4f7d6d5032b1a188f98e0c149ef6bf91e76af63e'
make: *** [orca] Error 1

解决方式

yum install gcc-c++

`bison’ is missing on your system.

错误信息

该问题可能出现在make阶段

***
ERROR: `bison' is missing on your system. It is needed to create the
file `gram.c'. You can either get bison from a GNU mirror site
or download an official distribution of PostgreSQL, which contains
pre-packaged bison output.
***
make[3]: *** [gram.c] Error 1
make[3]: Leaving directory `/home/gpdb_src/gpdb-5.16.0/src/backend/parser'
make[2]: *** [parser/gram.h] Error 2
make[2]: *** Waiting for unfinished jobs....
AWK='gawk' /bin/sh Gen_fmgrtab.sh pg_proc_combined.h.tmp
make[3]: Leaving directory `/home/gpdb_src/gpdb-5.16.0/src/backend/utils'
make[2]: Leaving directory `/home/gpdb_src/gpdb-5.16.0/src/backend'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/home/gpdb_src/gpdb-5.16.0/src'
make: *** [all] Error 2

解决方式

# 同样的flex也可能出现类似问题，解决稍微麻烦一点

yum install -y bison.x86_64 bison-devel.x86_64

#之后需要make clean，重新执行安装流程

openssl/hmac.h: No such file or directory

错误信息

In file included from include/reader.h:4:0,
                 from include/gpreader.h:4,
                 from src/gpreader.cpp:1:
include/s3common_headers.h:8:26: fatal error: openssl/hmac.h: No such file or directory
 #include <openssl/hmac.h>
                          ^
compilation terminated.