.. _userdb_text_config: UserDBConfig Text (Legacy) Config Format ======================================== This is a tool to generate the configurations for all the users across both :ref:`cu` and :ref:`mptgs` so that they will preserve desired statistical properties across the different application/traffic types. Installation ------------ UserDBConfig is a collection of Perl scripts and modules. To use it some additional modules are required (that are part of a defualt setup). The tar that includes UserDBConfig includes the extra modules it needs so they can be installed without an Internet connection; however, to build those modules requires a general development environment. The module build/install process requires ``gcc`` and the ability to process Perl Makefiles (e.g., perl-ExtUtils-MakeMaker). Once those system requirements are set change into the ``libs`` directory. From there run ``./install_libs.sh`` which will process the modules there in the order givein in ``install_order``. Once all the libraries are installed, UserDBConfig is ready to run to generate its files. The tools to fetch ConsoleUser information requires ConsoleUser to be installed. There are a couple of tools it provides that are used, and they will also look for a file ``/usr/local/tg/db.info`` file which can store DB connection information instead of needing to repeat it on the command line for each fetch. Completeing the process requires a MySQL/MariaDB server to be available for :ref:`cu` configs. If the tool is generating :ref:`mptgs` as well a PostgreSQL database server is also required. The instructions below assume the MariaDB server is installed locally and that the user running the commands does not require a password. If either of those assumptions are not true for your environment please modify the commands accordingly. Fetching ConsoleUser configs expects that ConsoleUser has been installed in ``/usr/local/tg/cu``. Process ------- To generate the final configurations one must first setup the tool's configuration files. After that, ``GenerateTables.pl`` will produce intermediate files that can be used to populate a MySQL/MariaDB database. This database can be populated by using the output file ``populate_db.sql`` with a command like Those intermediate files can also populate a separate PostgreSQL database. This database is used directly by the :ref:`mptgs` during its operation. In contrast, the ``mysql`` database is used to coordinate distribution of :ref:`cu` configurations to appropriate controllers and directories. The 2 steps, then, to generate config files and populate the database for ConsoleUsers is .. code-block:: bash $ ./GenerateTables.pl main.config $ mysql < populate_db.sql There are additional steps to populate the database that MPTGS will use. That process has been put into the script ``doAllMPTGS.sh`` which will prepare a PostgreSQL database, run ``GenerateTables.pl`` massage the data for MPTGS needs, then populate the DB. After this process that database is ready to be used by an instance of MPTGS. ``doAllMPTGS.sh`` will populate the database named in the environment variable ``PGDATABASE`` if it exists, otherwise the name is hardcoded. To populate both the databases the following process is recommend: .. code-block:: bash $ ./doAllMPTGS.sh $ mysql < populate_db.sql Main Config ----------- This is the file that is passed to ``GenerateTables.pl`` and controls all of the parameters and relationships ``GenerateTables.pl`` is to produce. .. table:: Main Config Options +-----------------------+---------------------------------------------------+ | Key | Description | +=======================+===================================================+ | MAGIC | Must be the first line of the file | +-----------------------+---------------------------------------------------+ | NMODULES | The number of modules defined in this file. | +-----------------------+---------------------------------------------------+ | REQUIRED_MODULES | An int, the first this many modules are ones that | | | each user is required to have in their list. | +-----------------------+---------------------------------------------------+ | MODULE*N* | *N* starts with 1 and increases from there. Each | | | MODULE is one traffic type or activity module to | | | be configured. | +-----------------------+---------------------------------------------------+ | NDOMAINS | This is the number of logical groups the users are| | | divided into. They need not have any grouping | | | based on any IP or name domain. This is just so | | | that users can be grouped and relationships | | | between the groups specified. | +-----------------------+---------------------------------------------------+ | UDOMAIN*N* | *N* starts with 1 and increases from there. Each | | | of these points to a CSV of users who comprise | | | this group | +-----------------------+---------------------------------------------------+ | SDOMAIN | For some ConsoleUser apps this is where files | | | will be saved. | +-----------------------+---------------------------------------------------+ | MDIST | An :ref:`dist_matrix` representing email and other| | | communication method relationships between user | | | domains | +-----------------------+---------------------------------------------------+ | NODE_PARAM | A parameter that shapes the distributions used in | | | many random selections. | +-----------------------+---------------------------------------------------+ | NWDOMAINS | The number of "web" domains, which provide | | | services such as websites or ftp sites or similar.| +-----------------------+---------------------------------------------------+ | WDOMAIN*N* | *N* starts with 1 and increases from there. Each | | | of these points to a file (or really files) that | | | list the values a user could choose to interact | | | with the web (or other service) when trying to | | | generate this type of traffic to this logical | | | grouping of servers. | +-----------------------+---------------------------------------------------+ | WEB_PARAM | A parameter that shapes the distributions used in | | | many random selections. | +-----------------------+---------------------------------------------------+ | ZIPF | A parameter that affects the popularity | | | distribution sites within a domain. | +-----------------------+---------------------------------------------------+ | WDIST | An :ref:`dist_matrix` representing relationships | | | between users and the services available in a | | | "web" domain | +-----------------------+---------------------------------------------------+ | BLOG_DIST | An :ref:`dist_matrix` representing relationships | | | between users and the blog sites available in a | | | "web" domain | +-----------------------+---------------------------------------------------+ | FORUM_DIST | An :ref:`dist_matrix` representing relationships | | | between users and the forum sites available in a | | | "web" domain | +-----------------------+---------------------------------------------------+ | NFileSharingDIST | The number of "file sharing" domains, that is | | | groups file sharing sites in some logical grouping| +-----------------------+---------------------------------------------------+ | FileSharingDIST | An :ref:`dist_matrix` representing relationships | | | between users and the file sharing sites available| | | in a file sharing domain | +-----------------------+---------------------------------------------------+ | BABBLE | The default :ref:`babble_file` for ConsoleUsers | +-----------------------+---------------------------------------------------+ | TORRENT_PARAM | Paramater that shapes the distribution of remote | | | files a user can choose to download. | +-----------------------+---------------------------------------------------+ | TO_TORRENT_PARAM | Paramater that shapes the distribution of local | | | files a user can choose to share. | +-----------------------+---------------------------------------------------+ | TORRENT_ZIPF | A parameter that affects the popularity | | | distribution files one may download. | +-----------------------+---------------------------------------------------+ | TO_TORRENT_ZIPF | A parameter that affects the popularity | | | distribution files one may share. | +-----------------------+---------------------------------------------------+ | TRACKER_PARAM | Paramater that shapes the distribution of trackers| | | a user can use while sharing. | +-----------------------+---------------------------------------------------+ | TRACKER_ZIPF | A parameter that affects the popularity | | | distribution trackers users will chose from. | +-----------------------+---------------------------------------------------+ | TORRENT_LIST | A file listing remote files that users can | | | attempt to download. | +-----------------------+---------------------------------------------------+ | TO_TORRENT_LIST | A file listing local files that users can attempt | | | to share. | +-----------------------+---------------------------------------------------+ | TRACKER_LIST | A file listing trackers that users can attempt to | | | use. | +-----------------------+---------------------------------------------------+ | TRANSMISSION_MIN_SEED | The minimum number of transmission users who will | | | seed each file from the TORRENT_LIST | +-----------------------+---------------------------------------------------+ | TRANSMISSION_MAX_SEED | The maximum number of transmission users who will | | | seed each file from the TORRENT_LIST | +-----------------------+---------------------------------------------------+ General Configuration ^^^^^^^^^^^^^^^^^^^^^ Lines in this file are of the format ``key value`` with whitespace seperating the key and the value. The interpretation of the value is dependent on the key. Sometimes it is a numeric value, sometimes it is the name of a file, sometimes it is a matrix of relationships. The only key that does not follow this is ``MAGIC`` which must be alone on the first line to indicate to the tool that this is a suitable configuration file. Lines that begin with ``#`` are comments, and are not consumed by the tool.