@@ -0,0 +1,150 @@ | |||
------------------------------- | |||
Project: EuryBOX | |||
Version: 1.0.0 | |||
Date: 2016/04/02 | |||
------------------------------- | |||
Author(s): | |||
Guillaume REMBERT, guillaume.rembert@euryecetelecom.com | |||
------------------------------- | |||
License: | |||
MIT - Please read LICENSE file for more information | |||
------------------------------- | |||
------------------------------- | |||
README INDEX | |||
------------------------------- | |||
0 - Project Presentation | |||
------------------------------- | |||
This section presents the EuryBOX project. | |||
Start first by reading these lines. | |||
------------------------------- | |||
1 - Repository Structure | |||
------------------------------- | |||
This section presents the repository structure. | |||
Read these lines to understand the files layout. | |||
------------------------------- | |||
2: Integration Methodology | |||
------------------------------- | |||
This section presents a methodology to be used to integrate the solution. | |||
Read these lines to check and/or define your integration plan. | |||
------------------------------- | |||
3: Development Guidelines | |||
------------------------------- | |||
This section presents coding style conventions. | |||
Read these lines to modify and/or extend the system. | |||
------------------------------- | |||
4: Quick Start | |||
------------------------------- | |||
This section summarizes what is needed to setup and run the solution. | |||
Read these lines to build, install and start the system. | |||
------------------------------- | |||
------------------------------- | |||
PART 0 - PROJECT PRESENTATION | |||
------------------------------- | |||
FIXME: TBD / Features, supported platforms, etc | |||
------------------------------- | |||
------------------------------- | |||
PART 1 - REPOSITORY STRUCTURE | |||
------------------------------- | |||
\ cfg (ConFiGuration): configuration files | |||
------------------------------- | |||
\ hw (HardWare): TBD | |||
\ sw (SoftWare): TBD | |||
------------------------------- | |||
\ doc (DOCumentation): documentation | |||
------------------------------- | |||
\ ddd (Detailed Design Document): architecture, algorithms and design details | |||
\ icd (Interface Control Document): API, internal and external files formats description | |||
\ trp (Test RePorts): stable releases test reports | |||
\ ugd (User GuiDe): installation, configuration and use guide | |||
------------------------------- | |||
\ hw (HardWare): hardware data | |||
------------------------------- | |||
\ bom (Bill Of Materials) | |||
\ dts (DaTaSheets) | |||
------------------------------- | |||
\ sw (SoftWare): software data | |||
------------------------------- | |||
\ bin (BINaries): qualified binaries | |||
\ RELEASE_VERSION_X.XX.XX | |||
\ SOFTWARE_NAME-x86_64_X.XX.XX-gcc_X.XX.XX-glib_X.XX.XX | |||
\ pkg (PacKaGes): platform specific installation packages | |||
\ RELEASE_VERSION_X.XX.XX | |||
\ SOFTWARE_NAME | |||
\ apk: Android | |||
\ bsd: BSD Unix | |||
\ deb: Debian/Ubuntu Linux | |||
\ ios: iOS | |||
\ rpm: RedHat/CentOS/Fedora/OpenSUSE Linux | |||
\ win: Windows | |||
\ src (SouRCes): software source code | |||
\ administration_center | |||
\ core | |||
\ backup | |||
\ destruction | |||
\ management | |||
\ restoration | |||
\ scheduling | |||
\ transfer | |||
\ creation_center | |||
\ core | |||
\ build | |||
\ configuration | |||
\ test | |||
\ version | |||
\ applications | |||
\ communication | |||
\ finances | |||
\ planning | |||
\ research | |||
\ resources | |||
\ security | |||
\ support | |||
\ exploitation_center | |||
\ core | |||
\ diagnostic | |||
\ deployment | |||
\ installation | |||
\ inventory | |||
\ maintenance | |||
\ monitoring | |||
\ reparation | |||
\ update | |||
\ quality_center | |||
\ core | |||
\ audit | |||
\ reproduction | |||
\ validation | |||
------------------------------- | |||
\ tst (TeST): test tools and data-sets | |||
------------------------------- | |||
\ sh (SHell) | |||
\ eurybox_auto_test | |||
------------------------------- | |||
------------------------------- | |||
PART 2: INTEGRATION METHODOLOGY | |||
------------------------------- | |||
FIXME: TBD / Specs -> Design -> Code -> Test | |||
------------------------------- | |||
------------------------------- | |||
PART 3: DEVELOPMENT GUIDELINES | |||
------------------------------- | |||
FIXME: TBD / variables / functions / comments / ... | |||
------------------------------- | |||
------------------------------- | |||
PART 4: QUICK START | |||
------------------------------- | |||
FIXME: TBD / HW and SW requirements, OS setup, build, install, test | |||
------------------------------- | |||
@@ -0,0 +1,150 @@ | |||
#!/bin/bash | |||
#EuryBOX configuration file | |||
#2015/06/25: Guillaume REMBERT, initial version based on EuryBOX prototype | |||
################################### | |||
### SCRIPT EXECUTION PARAMETERS ### | |||
################################### | |||
EURYBOX_CORPORATE="My Corporate" | |||
#Details level on script execution | |||
#Can be: debug, verbose, info, warning or error | |||
EURYBOX_LOG_LEVEL="verbose" | |||
#Parallelism depth / depends on host CPU | |||
EURYBOX_PARALLELISM_LEVEL="8" | |||
#Max retry on failure before stopping execution | |||
EURYBOX_MAX_RETRY_ON_FAILURE="5" | |||
#Network calls timeout (in seconds) | |||
EURYBOX_NETWORK_TIMEOUT="1" | |||
############################## | |||
### ENVIRONMENT PARAMETERS ### | |||
############################## | |||
#Can be: centos (6.x | 7.X), redhat (6.x | 7.X) #TO BE TESTED: fedora, opensuse, ubuntu, debian, archlinux, gentoo | |||
EURYBOX_DISTRIB_TYPE="centos" | |||
EURYBOX_DISTRIB_VERSION="7.2" | |||
EURYBOX_DISTRIB_MAC_TYPE="selinux" | |||
EURYBOX_DOMAIN="mydomain.mytld" | |||
#Hypervisor parameters | |||
#Type = xen, qemu (with and without kvm) | |||
EURYBOX_HYPERVISOR=( | |||
[TYPE]="qemu" | |||
) | |||
#Services parameters | |||
#Disks are autodetected | |||
#Supported OS yet (for update): archlinux, centos5, centos6, centos7, debian6, debian7, debian8, fedora20, fedora21, fedora22, fedora23, gentoo, redhat5, redhat6, redhat7, ubuntu12, ubuntu14, ubuntu16 | |||
declare -A EURYBOX_SERVICES | |||
EURYBOX_SERVICES=( | |||
[1,NAME]="gw001m" | |||
[1,HOST]="192.168.123.133" | |||
[1,PORT]="22" | |||
[1,USER]="root" | |||
[1,DISK]="/vm_storage/images/gw001m_hdd1.qcow2" | |||
[1,OS]="openbsd5" | |||
## [1,SERV]="GATEWAY" | |||
## [1,TYPE]="pf" | |||
[2,NAME]="srv001m" | |||
[2,HOST]="srv001m.$EURYBOX_DOMAIN" | |||
[2,PORT]="22" | |||
[2,USER]="root" | |||
[2,DISK]="/vm_storage/images/srv001m_hdd1.qcow2 /vm_storage/images/srv001m_hdd2.qcow2 /home/vm_store/service001b.qcow2" | |||
[2,OS]="centos7" | |||
## [2,SERV]="DHCP" | |||
## [2,TYPE]="dhcpd" | |||
) | |||
#Services number (linked to the number of parameters per service) | |||
#Should not have to be changed | |||
EURYBOX_SERVICES_NUMBER=`expr ${#EURYBOX_SERVICES[@]} / 6` | |||
######################### | |||
### BACKUP PARAMETERS ### | |||
######################### | |||
declare -A EURYBOX_BACKUP_DESTINATION | |||
declare -A EURYBOX_BACKUP_TARGETS | |||
declare -A EURYBOX_BACKUP_ARCHIVE | |||
#Execute a cold backup | |||
EURYBOX_BACKUP_COLD=true | |||
#Execute a hot backup | |||
EURYBOX_BACKUP_HOT=false | |||
#Low disk space threshold (Kilo-Bytes) | |||
EURYBOX_BACKUP_LOW_WARNING_LEVEL="250000000" | |||
EURYBOX_BACKUP_LOW_CRITICAL_LEVEL="25000000" | |||
#Folder used for temporary operations | |||
EURYBOX_BACKUP_TMP_FOLDER="/tmp/bkp" | |||
#Pre-Backup Commands | |||
#Post-Backup Commands | |||
#Backup targets (in addition to services attached disks backup and additionnal disks from config) | |||
EURYBOX_BACKUP_TARGETS=( | |||
[0]="/var/log" | |||
[1]="/etc" | |||
) | |||
#Backup archive | |||
#FORMAT= tar/tar.gz/tar.bz2 | |||
#HASH= sha256/sha512 | |||
#ECC= par2 | |||
#ECC_LEVEL = 0/1/.../99/100 | |||
#ENC_ALGO= openssl supported algo (aes-256-xts/aes-256-ctr/...) | |||
EURYBOX_BACKUP_ARCHIVE=( | |||
[NAME]="eurybox_archive" | |||
[MASK]="400" | |||
[LABEL]="backup_u:object_r:backup_t:s0" | |||
[FORMAT]="tar.bz2" | |||
[HASH]="sha512" | |||
[ECC]="par2" | |||
[ECC_LEVEL]="10" | |||
[ENCRYPT]="true" | |||
[ENC_ALGO]="aes-256-ctr" | |||
[PASSWORD]="PutYourStrongPasswordHere" | |||
) | |||
#Backup destination | |||
#PROTOCOL= SSH/NFS | |||
EURYBOX_BACKUP_DESTINATION=( | |||
[PROTOCOL]="NFS" | |||
[HOST]="10.10.10.10" | |||
[PATH]="/mnt/backup/eurybox_archives" | |||
## [PORT]="2049" | |||
#FOR NFS ONLY => poRT 2049 : man nfs | |||
[TYPE]="nfs4" | |||
[MOUNT]="/mnt/local_backup_folder" | |||
#FOR SSH ONLY | |||
[USER]="root" | |||
[PORT]="22" | |||
) | |||
######################### | |||
### UPDATE PARAMETERS ### | |||
######################### | |||
############################### | |||
### EXEC COMMAND PARAMETERS ### | |||
############################### | |||
declare -A EURYBOX_EXEC_COMMANDS | |||
#Command to be executed on all VM (OS type discrimination possible) | |||
EURYBOX_EXEC_COMMANDS=( | |||
[0,CMD]="hostname" | |||
[0,OS]="all" | |||
[1,CMD]="yum -y install wget" | |||
[1,OS]="centos7" | |||
[2,CMD]="ping -c 1 www.$EURYBOX_DOMAIN" | |||
[2,OS]="all" | |||
) | |||
EURYBOX_EXEC_COMMANDS_NUMBER=`expr ${#EURYBOX_EXEC_COMMANDS[@]} / 2` |
@@ -0,0 +1,102 @@ | |||
#!/bin/bash | |||
#2015/06/25: Guillaume REMBERT | |||
#EuryBOX backup script | |||
START_TIME=$(date +%s) | |||
######################### | |||
# SCRIPT INITIALISATION # | |||
######################### | |||
#Get configuration variables and functions | |||
SCRIPT_COMMAND=$(readlink -f "$0") | |||
SCRIPT_ARGUMENTS="$@" | |||
SCRIPT_PATH=$(dirname "$SCRIPT_COMMAND") | |||
#Should not be changed except if you changed sw folder layout | |||
SCRIPT_CONF_PATH=$SCRIPT_PATH/../../../../../cfg/sw/eurybox.conf | |||
SCRIPT_FUNC_PATH=$SCRIPT_PATH/../../../eurybox.functions | |||
#Check and load ressources | |||
if [[ -f $SCRIPT_CONF_PATH && -f $SCRIPT_FUNC_PATH ]] | |||
then | |||
source $SCRIPT_CONF_PATH | |||
source $SCRIPT_FUNC_PATH | |||
eurybox_message_display debug STATUS "Configuration and function scripts successfully sourced" | |||
else | |||
echo "[EURYBOX][ERROR][RESSOURCES] Configuration file $SCRIPT_CONF_PATH or functions file $SCRIPT_FUNC_PATH not found. Stopping execution" | |||
exit 1 | |||
fi | |||
eurybox_message_display message STATUS "Checking configuration" | |||
eurybox_check_configuration BACKUP | |||
#Check input parameters | |||
eurybox_message_display message STATUS "Checking parameters" | |||
eurybox_check_arguments $SCRIPT_ARGUMENTS | |||
#Check environment | |||
eurybox_message_display message STATUS "Checking environment" | |||
eurybox_check_environment BACKUP | |||
eurybox_message_display message STATUS "$EURYBOX_CORPORATE backup script started" | |||
############################### | |||
# VM + HYPERVISOR COLD BACKUP # | |||
############################### | |||
euryboxctrl_prepare_backup | |||
#Performing a cold backup | |||
if [ "$EURYBOX_BACKUP_COLD" = true ] | |||
then | |||
eurybox_message_display message STATUS "VM Cold backup started" | |||
#Shutdown all services vm | |||
euryboxctrl_shutdown_services_ssh | |||
euryboxctrl_check_all_services_halted | |||
#Create archive file | |||
euryboxctrl_create_arch | |||
#Re-Start all VMs | |||
euryboxctrl_start_services | |||
#Create errors correcting codes files | |||
euryboxctrl_create_eccf | |||
#Create description file | |||
euryboxctrl_create_desc | |||
#Create hash file | |||
euryboxctrl_create_hash | |||
#Create signature file | |||
euryboxctrl_create_sign | |||
eurybox_message_display message STATUS "Cold backup finished" | |||
fi | |||
################# | |||
# VM HOT BACKUP # | |||
################# | |||
#Performing a hot backup | |||
if [ "$EURYBOX_BACKUP_HOT" = true ] | |||
then | |||
eurybox_message_display warning STATUS "HOT BACKUP NOT AVAILABLE YET" | |||
#HDD | |||
#NB: use LVM snapshot for live backup... | |||
#NB: sync OS FS | |||
#NB: lock DB / prepare for bkp | |||
#CF on CentOS wiki / good KVM backup article/script example | |||
fi | |||
euryboxctrl_terminate_backup | |||
#Finished | |||
END_TIME=$(date +%s) | |||
DIFF_TIME=$(( $END_TIME - $START_TIME )) | |||
eurybox_message_display message STATUS "Backup finished" | |||
eurybox_message_display message TIMING "Backup took $DIFF_TIME seconds" |
@@ -0,0 +1,54 @@ | |||
#!/bin/bash | |||
#2015/06/27: Guillaume REMBERT | |||
#EuryBOX exec commands script | |||
START_TIME=$(date +%s) | |||
######################### | |||
# SCRIPT INITIALISATION # | |||
######################### | |||
#Get configuration variables and functions | |||
SCRIPT_COMMAND=$(readlink -f "$0") | |||
SCRIPT_ARGUMENTS="$@" | |||
SCRIPT_PATH=$(dirname "$SCRIPT_COMMAND") | |||
#Should not be changed except if you changed sw folder layout | |||
SCRIPT_CONF_PATH=$SCRIPT_PATH/../../../../../cfg/sw/eurybox.conf | |||
SCRIPT_FUNC_PATH=$SCRIPT_PATH/../../../eurybox.functions | |||
#Check and load ressources | |||
if [[ -f $SCRIPT_CONF_PATH && -f $SCRIPT_FUNC_PATH ]] | |||
then | |||
source $SCRIPT_CONF_PATH | |||
source $SCRIPT_FUNC_PATH | |||
eurybox_message_display debug STATUS "Configuration and function scripts successfully sourced" | |||
else | |||
echo "[EURYBOX][ERROR][RESSOURCES] Configuration file $SCRIPT_CONF_PATH or functions file $SCRIPT_FUNC_PATH not found. Stopping execution" | |||
exit 1 | |||
fi | |||
eurybox_message_display message STATUS "Checking configuration" | |||
eurybox_check_configuration BACKUP | |||
#Check input parameters | |||
eurybox_message_display message STATUS "Checking parameters" | |||
eurybox_check_arguments $SCRIPT_ARGUMENTS | |||
#Check environment | |||
eurybox_message_display message STATUS "Checking environment" | |||
eurybox_check_environment BACKUP | |||
eurybox_message_display message STATUS "$EURYBOX_CORPORATE exec commands script started" | |||
#Start all VMs | |||
euryboxctrl_start_services | |||
#Execution commands on virtual machines / services | |||
euryboxctrl_exec_command_services | |||
#Finished | |||
END_TIME=$(date +%s) | |||
DIFF_TIME=$(( $END_TIME - $START_TIME )) | |||
eurybox_message_display message STATUS "Command exec finished" | |||
eurybox_message_display message TIMING "Command exec took $DIFF_TIME seconds" |
@@ -0,0 +1,61 @@ | |||
#!/bin/bash | |||
#2016/04/22: Guillaume REMBERT | |||
#EuryBOX restoration script | |||
START_TIME=$(date +%s) | |||
######################### | |||
# SCRIPT INITIALISATION # | |||
######################### | |||
#Get configuration variables and functions | |||
SCRIPT_COMMAND=$(readlink -f "$0") | |||
SCRIPT_ARGUMENTS="$@" | |||
SCRIPT_PATH=$(dirname "$SCRIPT_COMMAND") | |||
#Should not be changed except if you changed sw folder layout | |||
SCRIPT_CONF_PATH=$SCRIPT_PATH/../../../../../cfg/sw/eurybox.conf | |||
SCRIPT_FUNC_PATH=$SCRIPT_PATH/../../../eurybox.functions | |||
#Check and load ressources | |||
if [[ -f $SCRIPT_CONF_PATH && -f $SCRIPT_FUNC_PATH ]] | |||
then | |||
source $SCRIPT_CONF_PATH | |||
source $SCRIPT_FUNC_PATH | |||
eurybox_message_display debug STATUS "Configuration and function scripts successfully sourced" | |||
else | |||
echo "[EURYBOX][ERROR][RESSOURCES] Configuration file $SCRIPT_CONF_PATH or functions file $SCRIPT_FUNC_PATH not found. Stopping execution" | |||
exit 1 | |||
fi | |||
eurybox_message_display message STATUS "Checking configuration" | |||
eurybox_check_configuration RESTORE | |||
#Check input parameters | |||
eurybox_message_display message STATUS "Checking parameters" | |||
eurybox_check_arguments $SCRIPT_ARGUMENTS | |||
#Check environment | |||
eurybox_message_display message STATUS "Checking environment" | |||
eurybox_check_environment RESTORE | |||
eurybox_message_display message STATUS "$EURYBOX_CORPORATE restoration script started" | |||
######################## | |||
# CLONE VM RESTORATION # | |||
######################## | |||
#Discovering available backups | |||
euryboxctrl_check_available_backups | |||
################################ | |||
# VM + HYPERVISOR RESTORATION # | |||
################################ | |||
#Finished | |||
END_TIME=$(date +%s) | |||
DIFF_TIME=$(( $END_TIME - $START_TIME )) | |||
eurybox_message_display message STATUS "Restoration finished" | |||
eurybox_message_display message TIMING "Restoration took $DIFF_TIME seconds" |
@@ -0,0 +1,59 @@ | |||
#!/bin/bash | |||
#2015/06/25: Guillaume REMBERT | |||
#EuryBOX update script | |||
START_TIME=$(date +%s) | |||
######################### | |||
# SCRIPT INITIALISATION # | |||
######################### | |||
#Get configuration variables and functions | |||
SCRIPT_COMMAND=$(readlink -f "$0") | |||
SCRIPT_ARGUMENTS="$@" | |||
SCRIPT_PATH=$(dirname "$SCRIPT_COMMAND") | |||
#Should not be changed except if you changed sw folder layout | |||
SCRIPT_CONF_PATH=$SCRIPT_PATH/../../../../../cfg/sw/eurybox.conf | |||
SCRIPT_FUNC_PATH=$SCRIPT_PATH/../../../eurybox.functions | |||
#Check and load ressources | |||
if [[ -f $SCRIPT_CONF_PATH && -f $SCRIPT_FUNC_PATH ]] | |||
then | |||
source $SCRIPT_CONF_PATH | |||
source $SCRIPT_FUNC_PATH | |||
eurybox_message_display debug STATUS "Configuration and function scripts successfully sourced" | |||
else | |||
echo "[EURYBOX][ERROR][RESSOURCES] Configuration file $SCRIPT_CONF_PATH or functions file $SCRIPT_FUNC_PATH not found. Stopping execution" | |||
exit 1 | |||
fi | |||
eurybox_message_display message STATUS "Checking configuration" | |||
eurybox_check_configuration UPDATE | |||
#Check input parameters | |||
eurybox_message_display message STATUS "Checking parameters" | |||
eurybox_check_arguments $SCRIPT_ARGUMENTS | |||
#Check environment | |||
eurybox_message_display message STATUS "Checking environment" | |||
eurybox_check_environment UPDATE | |||
eurybox_message_display message STATUS "$EURYBOX_CORPORATE update script started" | |||
#Make sure all VM are started | |||
euryboxctrl_start_services | |||
#Update local repository | |||
euryboxctrl_update_local_repository | |||
#Update the physical machine | |||
euryboxctrl_update_hypervisor | |||
#Update virtual machines / services | |||
euryboxctrl_update_services | |||
#Finished | |||
END_TIME=$(date +%s) | |||
DIFF_TIME=$(( $END_TIME - $START_TIME )) | |||
eurybox_message_display message STATUS "Update finished" | |||
eurybox_message_display message TIMING "Update took $DIFF_TIME seconds" |