Fermilab


MINOS Offline Documentation
[home] [installing MINOS software] [supported platforms] [package links] [mailing list] [HyperNews] [FAQ] [computing help] [MINOS glossary] [archives]

Installing External Packages for MINOS




INTRODUCTION

The Minos analysis system requires access to a database holding the constants required for analysis. Collaborators away from Fermilab or the detectors are unlikely to relish the idea of getting constants over the internet from the central store at Fermilab, so there is a way to have constants served locally, at collaborating institutions. The scheme is to encourage each collaborating institution to host its own "site-database", with all the data needed to do whatever analysis is of interest at that institution. This will undoubtedly cut down on network traffic, and thus speed up analysis at institutions; however it requires institutions with no or very limited support to have their own database management systems.

The following is instruction how to install external packages required for Database Interface. All packages will be installed in the $INSTALLATION directory. This installation does not require admin previliges. If you have such privileges I recommend installing all external packages in the "/usr" area. To do so, you need to substitute all occurrences of $INSTALLATION in this installation with "/usr".

As of Frozen Release R1.26 (Snapshot Release S07-01-30-R1-25)The only necessary external product to be installed is:
  1. MySQL: client and server
and if you are running a mainstream platform such as Scientific Linux > you may have it already and your job is done!

For earlier releases the three necessary external products need to be installed in the order listed:

  1. unixODBC Driver Manager
  2. MySQL: client and server
  3. MyODBC driver

Tar files of the recommended version of the external packages can be found at:

http://www-numi.fnal.gov/offline_software/external_pkgs/tar_files/RECOMMENDED/

One will find source for unixODBC, mysql, MyODBC.

After installing the packages you can:



INSTALLING unixODBC DRIVER MANAGER

Only necessary for R1.25 and before.

  1. Download the source tar file for unixODBC
    (see above for location of recommended version, or http://www.unixODBC.org/download.html if you're bold).

  2. Decompress the tar file using gunzip prompt> gunzip unixODBC-x.x.x.tar.gz prompt> tar -xvf unixODBC-x.x.x.tar Note: On Linux, the commands gunzip and tar can both be executed at once by using prompt> tar -zxvf unixODBC-x.x.x.tar.gz
  3. prompt> cd unixODBC-x.x.x
  4. The options "--disable-gui" and "--disable-drivers" are used to create minimal configuration. prompt> ./configure --prefix=$INSTALLATION \ --disable-gui \ --disable-drivers
  5. On MacOSX only (as described in file README.OSX): Edit the file libtool in the unixODBC base directory and find the line: whole_archive_flag_spec="-all_load \$convenience" Replace the line with: whole_archive_flag_spec=
  6. To compile it just type: prompt> gmake
  7. To install type: prompt> gmake install

    After installation libodbc.so and libodbcinst.so will be placed in $INSTALLATION/lib directory. sql.h and sqlext.h will be in $INSTALLATION/include. (MacOSX will also install libodbc.dylib and libodbcinst.dylib in $INSTALLATION/lib.)

Note: it is possible to install other compatible libraries (such as iODBC). If these libraries go by different names you can specify this to the MINOSSOFT build system by setting UNIXODBCNAME. For example, to iodbc if using the iODBC libraries.


INSTALLING unixODBC on Debian GNU/Linux

Unixodbc v2.0.8 is currently part of Debian's "unstable" distribution (Sid) and will likely perculate down to "testing" soon.

  1. Make sure "unstable" is in your /etc/apt/sources.list file: with a line like: deb http://ftp.us.debian.org/debian unstable main contrib non-free
  2. Then give: apt-get update apt-get install unixodbc



MySQL

The recommended database engine is MySQL (available at http://www.mysql.com). MySQL is a completely free product with an extensive user-base, and a fairly long history. Third-party manuals atre available from both O'Reilly (Yarger, Reese and King: mSQL and MySQL) and New Riders Press (DuBois: MySQL). MySQL is supplied as part of recent RedHat distributions, and is available as rpm, binary-tarball, or source code. This page has instructions for installing MySQL from source code, rpm, and on Debian GNU/Linux. The binary-tarball installation only comes with static (.a) libraries so you should not install from binary-tarball.

Installing from RPM is by far the easiest way to install MySQL, but this requires root access and write permissions for /usr/lib. Installing from source code is more complicated but does not require root access, only a tar file available from MySQL. Our current (R1.26 and greater) interface requires version 4.1 or later and as of this writing, the recommended version of MySQL is 5.0.22


INSTALLING MySQL WITH RPM

The easiest way to install a MySQL server is from rpm format, which requires root access to the machine hosting the database and the permissions to write to /usr/local/.
  1. Download the files from MySQL Downloads (http://www.mysql.com/downloads/index.html). A minimal system requires four rpms: MySQL-5.0.xx.i386.rpm MySQL-client-5.0.xx.i386.rpm MySQL-devel-5.0.xx.i386.rpm MySQL-shared-5.0.xx.i386.rpm
  2. To install MySQL, download the above files (the 5.0.xx is a version number and may change in the future; Minos standard is 5.0.xx). To install, enter: prompt> rpm -Uvh MySQL-5.0.xx.i386.rpm prompt> rpm -Uvh MySQL-client-5.0.xx.i386.rpm prompt> rpm -Uvh MySQL-devel-5.0.xx.i386.rpm prompt> rpm -Uvh MySQL-shared-5.0.xx.i386.rpm

    You should see some ### marks as it is installed, followed by a note reminding you to change the root password.

  3. If you aren't sure that the packages installed correctly, you can check by typing: prompt> rpm -qa | grep -i mysql

    This will find all installed rpms with the phrase "mysql" or "MySQL" in their titles. This is also a way to see if MySQL has already been installed, and to check the version if it has been.

  4. Upgrading an old version of MySQL: If you find an old (pre-4.0) version of MySQL on your machine, you should probably upgrade it to 4.0. This is easy with rpm, just use the same commands as specified above. The "U" in -Uvh stands for "upgrade" and will upgrade an existing version if there is one, or install a new version if there isn't.

Some versions of rpm will happily install packages with both capitalizations (MySQL and mysql) at the same time. The uncapitalized versions seem to work all right, and if you have an install of mysql version 4.0, you should probably leave well enough alone.

Once the rpms are installed correctly, you may skip down to "AFTER INSTALLING MySQL" below unless installing for R1.25 and before in which case skip down to "Installing MyODBC" below. Note that the instructions use an environment variable called "INSTALLATION", which is not defined automatically by the rpm install. If you installed with rpm, executable programs will be in /usr/bin, and the database data directory will be /var/lib/mysql. Substitute /usr for INSTALLATION in the notes below.




COMPILING AND INSTALLING MySQL FROM SOURCE FILES: SERVER AND CLIENT

  1. If you do not have root access to your machine, you will have to compile a source version of MySQL. Download the source tar file for MySQL
    See above for location of recommended version. Alternatively, the bold and daring can look to http://www.mysql.com/downloads/index.html; look for the "Source downloads for XXX" part of the page and get "Tarball".
  2. prompt> gunzip mysql-5.0.xx.tar.gz prompt> tar -xvf mysql-5.0.xx.tar
  3. prompt> cd mysql-5.0.xx

  4. To compile it and install it in $INSTALLATION directory just type prompt> ./configure --prefix=$INSTALLATION \ --without-bench \ --enable-thread-safe-client \ --enable-shared \ --disable-static \ --enable-local-infile
  5. prompt> gmake
  6. prompt> gmake install prompt> cd $INSTALLATION/lib prompt> ln -s mysql/libmysqlclient* .
  7. Install the databases by executing (note that $INSTALLATION/bin/mysql_install_db by itself may not work!). prompt> cd $INSTALLATION prompt> ./bin/mysql_install_db
  8. If something goes wrong check MySQL docs (http://www.mysql.com/doc/I/n/Installing.html) or the MINOS FAQ for help.

    If you are using Irix and something goes wrong check MySQL notes for SGI-Irix (http://www.mysql.com/documentation/mysql/bychapter/manual_Installing.html#SGI-Irix) or view questions and answers in the MINOS FAQ for help.




INSTALLING MySQL FROM BINARY-TARBALL

The binary-tarball installation only comes with static (.a) libraries so you should not install from binary-tarball.


INSTALLING MySQL On Debian GNU/Linux

  1. Install the packages: apt-get install mysql-client mysql-server libmysqlclient10-dev The ``-dev'' package is needed for MYODBC. Follow the few setup pages. They are self explanatory

  2. As directed read /usr/share/doc/mysql-server/README.Debian. It tells you to do:

    • /usr/bin/mysqladmin -u root password "your-password"

    • cat > /root/.my.cnf [mysqladmin] user = root password = your-password <Control-D> chmod 600 /root/.my.cnf

  3. Test by doing: /etc/init.d/mysql stop /etc/init.d/mysql start You will now have the mysql server running and it will be automatically restarted next boot.

  4. Now, when you come to them, IGNORE the following subsections in the section "AFTER INSTALLING MySQL", below:
    • STARTING THE SERVER
    • SETTING UP THE SERVER


INSTALLING MyODBC DRIVER

Only necessary for R1.25 and before.

  1. Download the source tar file for MyODBC (release 3.51.09 or above)
    (see above for location of recommended version, or the page Source for MyODBC on Unix (http://www.mysql.com/downloads/api-myodbc.html)).
  2. prompt> gunzip MyODBC-3.51.09.tar.gz prompt> tar -xvf MyODBC-3.51.09.tar
  3. prompt> cd MyODBC-3.51.09
  4. Configure MyODBC for your installation
    • MySQL installed from source prompt> ./configure --prefix=$INSTALLATION \ --with-unixODBC=$INSTALLATION \ --with-mysql-path=$INSTALLATION \ --enable-thread-safe
    • MySQL installed from RPM or from Debian packages prompt> ./configure --prefix=/usr/local \ --with-unixODBC=/usr \ --with-mysql-path=/usr \ --enable-thread-safe
  5. prompt> gmake If the build fails with:-
      /usr/bin/ld: cannot find -lXext
    
    then possible solutions that work are:-

    • If you don't need any GUI interface (and the off-line code doesn't) configure with:-
      --disable-gui
      
    • Alternatively, to build the GUI, add the following soft links under /usr/lib/qt-3.1/lib (or whatever version QT_LDFLAGS is set to in config.log file):-
      libX11.so -> /usr/X11R6/lib/libX11.so.6.2
      libXext.so -> /usr/X11R6/lib/libXext.so.6.4
      
  6. prompt> gmake install
  7. Fix up soft links.

    On all platforms (except MacOSX)

    prompt> cd $INSTALLATION/lib prompt> ln -s libmyodbc3.so libmyodbc.so prompt> ln -s libmyodbc3.la libmyodbc.la On MacOSX only (as described in INSTALL file): prompt> cd driver/.libs prompt> gcc -bundle -flat_namespace -undefined dynamic_lookup \ -bind_at_load -o libmyodbc3-3.51.09.so *.o \ -L$INSTALLATION/lib -lodbcinst -lmysqlclient_r \ -lz -lc -lpthread prompt> cp libmyodbc3-3.51.09.so $INSTALLATION/lib prompt> cd $INSTALLATION/lib prompt> ln -s libmyodbc3-3.51.09.so libmyodbc3.so prompt> ln -s libmyodbc3.so libmyodbc.so prompt> ln -s libmyodbc3.la libmyodbc.la prompt> ln -s libmyodbc3.dylib libmyodbc.dylib
  8. For MySQL installed from RPM or from Debian packages:

    If not already there, add /usr/local/lib to /etc/ld.so.conf and run ldconfig.




REBUILDING ROOT

If ROOT was built on your system before MySQL was installed it will build without MySQL support and needs to be rebuilt.


AFTER INSTALLING MySQL




STARTING THE SERVER

You should add the following directory to your PATH: $INSTALLATION/bin Start up the database server by entering (note that $INSTALLATION/bin/mysqld_safe by itself may not work!): prompt> cd $INSTALLATION prompt> ./bin/mysqld_safe --local-infile=1 & (safe_mysqld prior to mysql 4) This starts the server, not the client, so this will not bring up a mysql prompt.
To check that the server is running, do a prompt> ps -A | grep -i mysqld This should display one instance of mysqld_safe and one to three of mysqld. costello> ps -A | grep -i mysqld 31395 pts/2 00:00:00 mysqld_safe (or safe_mysqld prior to mysql 4) 31411 pts/2 00:00:00 mysqld 31413 pts/2 00:00:00 mysqld 31414 pts/2 00:00:00 mysqld

If you are having problems starting your MySQL server, go to MySQL helps and howtos.

Getting MySQL to startup on reboots: Every time the computer reboots, it should restart MySQL with a command like the one above. If you have installed from rpm files, you should have the correct files to start mysql already ( the files are in /etc/rc.d/rc3.d/S90mysql or /etc/rc.d/rc6.d/S90mysql on a RedHat 6.2 system ). If you did not install from rpms, you should add a line to the startup procedure; on a RedHat 6.2 system which boots up in runlevel 3, this should go into /etc/rc.d/rc3.d/S99local as the last two lines ( note that changing the startup files requires root access ): cd $INSTALLATION ./bin/ mysqld_safe --local-infile=1 & (safe_mysqld prior to mysql 4)


SETTING UP THE SERVER

After starting the server, change the database root password as soon as possible. Do this by typing (be sure you have the $INSTALLATION/bin directory in your path): prompt> mysqladmin --user=root password <root_pw>

where <root_pw> is the new password- it prompts for the new password, which you must enter twice. Note that there are no quotes around the password. PLEASE MAKE SURE YOU WRITE DOWN THE MYSQL ROOT PASSWORD. However, if you do ever loose it you can still get back in so long as you can run under the same account as the server by Resetting the Root Password

It wont let me change the root password!??! Sometimes there is a strange problem which requires specifying the host ip name to change the root password. Try:

prompt> mysqladmin --user=root --host=198.137.240.92 password <root_pw> where you substitute the host machine ip number for 198.137.240.92

MINOS STANDARD ACCOUNTS AND DATABASES

The Minos system requires several database accounts to be set up (a "database account" is distinct from a regular "computer account"). There is a generic account which is only allowed to read data, and cannot change anything. The current scheme for implementing flat-file reading requires an installation of MySQL, so there is an account which is able to make entries in a special database set up for temporary files only. Entering data is done with a special account with its own password, and eventually there will be an account used to move data around in the collaboration.

CREATING DATABASES

In MySQL terms, a "database" is a series of related tables. Each "instance" of the server can have several "databases", and each "database" can have many "tables" (each table has several columns and many rows). Creating a new database can be done from the mysql> prompt or through mysqladmin. Creating new tables in an existing database is done through the mysql> prompt.

MINOS STANDARD DATABASES

As of this writing (May 2, 2001) the Minos system requires every MySQL installation to have three databases installed: the mysql database, which stores access data and is generated automatically during the MySQL install; a temp database for reading text files, and a offline (formerly caldet) database which stores all the data for the caldet system. To create the temp and offline databases: prompt> mysql --user=root --password=<root_pw> Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 7 to server 3.23.37 Type 'help;' or '\h' for help. Type '\c' to clear the buffer mysql> create database temp; mysql> create database offline;

MINOS STANDARD ACCOUNTS

Every connection to MySQL requires a database account, which is a separate entity from a normal computer account. Every MySQL database has its own set of allowed accounts. It is possible to restrict account access dramatically- one can set limits on which IP numbers a connection is allowed to originate from, which tables a request may access, and even which columns of which tables an account may access (read, write, delete and change privileges are all controllable seperately). This is all controlled by logging in as root and issuing GRANT commands. This note will only scratch the surface of this topic.

The standard accounts for Minos are:

Account Name Passwd Database Comments ------------ ------ -------- -------- reader <pw> temp access to make temp tables only reader <pw> *(except temp) read access only, can't change anything writer <pw> * full access to minos databases Entering these accounts is done by logging into the mysql prompt as root and issuing GRANT commands: prompt> mysql --user=root --password=<root_pw> Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 7 to server 3.23.37 Type 'help;' or '\h' for help. Type 'c' to clear the buffer mysql> grant select on offline.* to reader@<HOST> identified by "<reader_pw>"; mysql> grant ALL on temp.* to reader@<HOST>; mysql> grant ALL on offline.* to writer@<HOST> identified by "<writer_pw>"; mysql> grant ALL on temp.* to writer@<HOST>; mysql> flush privileges; Privilege specifiers:
SELECT
allows user to retrieve existing data from tables
ALL
gives user all privileges besides grant privileges
Possibilities for <HOST>:
"%.[domain_name]"
ie reader@"%.slac.stanford.edu" or temper@"%.fnal.gov gives access from every computer within that domain name
"hep%.university.edu"
would allow access from ie reader@hep01.university.edu and reader@hep02.university.edu
This sort of <HOST> could be used for a group of similarly named computers, all of which should have access to the MySQL database.
Note: Changes to the privileges tables do not take effect until the "flush privileges" command has been executed. If you don't flush the privileges, the new accounts will not be usable...

Also note: Granting ALL on *.* is a very dangerous thing to do, as the account gets privileges you might not expect, like the ability to shut down the server! Granting ALL on offline.* is much more benign.


CHECKING THE SERVER

Test your installation by trying to connect to some MySQL server with your installation of the mysql program. prompt> mysql -u <user_name> -h <mysql_server> -p where <user_name> is the name of a user to whom you have assigned privileges and <mysql_server> is the name of the machine running the server you are connecting to.


RUNNING THE CLIENT

Go to Test MySQL for more some things to do to with MySQL.

Now you should be able to open a client connection to the mysql server by typing prompt> mysql --user=root --password=<root_pw>

where <root_pw> is the new password (note: no quotes around either root or the password).

In MySQL you should see a prompt like Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 7 to server 3.23.37 Type 'help;' or '\h' for help. Type '\c' to clear the buffer mysql> If you enter (leaving off the trailing ; is a bad idea): mysql> use mysql; mysql> show tables; You should see a list of the access tables +-----------------+ | Tables_in_mysql | +-----------------+ | columns_priv | | db | | func | | host | | tables_priv | | user | +-----------------+ 6 rows in set (0.04 sec) These tables control who is allowed to access what parts of the database. Access depends on where people are logged in from, as well, so it is possible to have an account which can connect from the local cluster but not from outside.


TELL THE WORLD

As a matter of courtesy, you should send the database group an email saying you have installed a server. The current database manager's email address is Nick West


Last Modified: $Date: 2007/07/18 12:34:02 $
Contact: minos_software_discussion@fnal.gov
Page viewed from http://www-numi.fnal.gov/offline_software/srt_public_context/WebDocs/install_products.html
Fermilab
Security, Privacy, Legal Fermi National Accelerator Laboratory