Jump to content
torndeco

extDB (arma3 extension linux/windows)

Recommended Posts

VERSION 11

  • Randomized Unique ID starting Number (not used in testapp to make testing easier)
  • Also reduced the range of used ids to 65536, more than enough, was 1-> max integer :P
  • Added DB_RAW_NO_EXTRA_QUOTES, doesn't add "" to varchar etc
  • Using CMake Build System for Windows Binarys

Edited by Torndeco

Share this post


Link to post
Share on other sites

Sstarting to get back into coding for extDB

Just noticed i had no documentation for MISC Class on it, so wiki got updated abit

https://github.com/Torndeco/extDB/wiki

Version 12

  • New Log Class, so u can output text to extDB logfiles. (note this isn't to replace diag_log, diag_log will be faster)
  • All Extensions now have basic startup + error logging
  • Debug Log Extension has ability to log DB queries (filter=0), performance hit
  • Fixed DB_BASIC get all commands
  • Added minor Error Handling Logic checks when adding Protocols i.e database connected + compatible

Important

Sqlite Databases are now moved (located) in the extDB/ directory

Most people seem to use MySQL so it shouldn't effect much people hopefully.

This is for next feature as i will use an sqlite database by deafult to cache Steam Querys for VAC Bans, and to help keep arma3 root directory neater.

edit:

Updated Linux Makefile

Linux Version now uses Intel TBB Malloc

Edited by Torndeco

Share this post


Link to post
Share on other sites

Just a heads up there is an issue with multi threading & sqlite for extDB...

THIS ISSUE DOESN'T EFFECT MYSQL

Temp workaround

Edit extdb-conf.ini

Change

Threads = 1

In database section

maxSessions = 1

You will also need to avoid SYNC calls to database... just use ASYNC calls with above settings.

Basicly we are avoiding any scenario with multi-threaded calls to sqlite.

----------------

----------------

Currently working on a solution

edit: Fixed in V13

Edited by Torndeco

Share this post


Link to post
Share on other sites

Version 13

  • CMake Build System Updates + Fixes
  • ADDED: MySQL compress config option
  • ADDED: WIP Rcon Code (only used in rcon test app atm)
  • UPDATED: DB_BASIC now contains Other 1 + Other 2 + Other 3 Fields
  • UPDATED: DB_BASIC now has logging output
  • FIXED: SQlite DB Locked concurrent access

Thanks to firefly2442 who been helping out & doing most of work with CMake Build System...

CMake now builds all versions of extension + various test applications

Linux Users

Linux now uses CMake Build System.

New experimental rcon (based off Bercon) test program, to build set

COMPILE_TEST_RCON=TRUE

./extDB-rcon 2302 password "SAY -1 Restart 5 minutes" &

Note the &

Atm the program has a timeout value roughly 30 seconds after sending command...

Its still wip, but should work for now for restart warnings

Edited by Torndeco

Share this post


Link to post
Share on other sites
Guest

Release frontpaged on the Armaholic homepage.

==================================================

You are not registered on Armaholic, or at least not that we are aware of. In the future we offer the possibility to authors to maintain their own pages.

If you wish to be able to do this as well please register on Armaholic and let me know about it.

This is not mandatory at all! Only if you wish to control your own content you are welcome to join, otherwise we will continue to follow your work like we have always done ;)

When you have any questions already feel free to PM or email me!

Share this post


Link to post
Share on other sites

I kinda, screwed up the logic in return values opps :butbut:

Basicly i had return values of

[0, "Error Message]
["Result"]

Instead of

[0, "Error Message]
[1, "Result"]

So i don't break anyone's code...

Added New Protocols (with correct output)

DB_RAW_V2

DB_BASIC_V2

etc...

I will add deprecated warnings to the logfiles on the weekend etc.

Also will leave the old Protocols in the extension for roughly the next month, before i remove them...

Its only a small change really sqf wise i.e adding a (select 1) for existing code.

Will update the wiki etc tomorrow...

-----------

Version 14

  • New Protocols V2
  • Changing Filesystem + Logging code -> Poco Library (smaller dll, should compile on debian now)
  • Log Filter Option changed -> Log Level look @ example file... (won't break anything will use default value if not set).

Share this post


Link to post
Share on other sites
Guest

Release frontpaged on the Armaholic homepage.

==================================================

You are not registered on Armaholic, or at least not that we are aware of. In the future we offer the possibility to authors to maintain their own pages.

If you wish to be able to do this as well please register on Armaholic and let me know about it.

This is not mandatory at all! Only if you wish to control your own content you are welcome to join, otherwise we will continue to follow your work like we have always done ;)

When you have any questions already feel free to PM or email me!

Share this post


Link to post
Share on other sites

Working on V15 atm....

Still some things i like to get done...

In the meantime there is a test build up atm @ https://github.com/Torndeco/extdb/tree/dev/windows_release/15%20Dev

Changes are so far

DB_PROCEDURE fixes (was broken for multiple inputs etc)...

If someone could test it that be great :)

DB_CUSTOM_V2

https://github.com/Torndeco/extdb/wiki/Calls:-Protocol--DB_CUSTOM_V2

It allows you to define the SQL Statements in a config file & number of inputs & if u like to run inputs / outputs through a sanitize code.

So that way u can freedom for SQL commands without exposing raw sql statements.

For example

[ADD_PLAYER]
SQL_1 = INSERT into Players VALUES ($INPUT_1, $INPUT_2,
SQL_2 = $INPUT_3, $INPUT_4);
Number of INputs = 4

[DEAD_PLAYER]
SQL_1 = UPDATE Players SET Alive = 0 WHERE UID = $INPUT_1;
Number of Inputs = 1

[GET_ALL_VEHICLE]
SQL_1 = SELECT * FROM VEHICLES;

Edited by Torndeco

Share this post


Link to post
Share on other sites

Version 15

Lot of various small / big things i be working on...

Been meaning todo update for abit

  • ADDED: MISC -> BEGUID, now u can get players BEGUID from their SteamID
  • ADDED: Deprecated Warnings in Log Files for V1 Protocols (be like a month or more before i remove them).
  • ADDED: DB_Custom_V2 Protocol allows u to define SQL commands to run in a foo.ini file.
    Basicly gives u freedom of Raw SQL commands without the added security risk, also option to use sanitize code or not.
    https://github.com/Torndeco/extdb/blob/stable/windows_release/15%20Debug%20Logging/extDB/db_custom/example.ini
  • UPDATED: Log Class you can now define the ID for the log entries (minor change really)
  • UPDATED: Sanitize Code now changed back to original code, exact same as DayZ Hive, less strict.
  • UPDATED: Rcon Test APP, is now a useable Rcon Chat Console App. Main use is for linux users for automated warnings. (Windows Users are got BEC / EPM Rcon)
    Windows Test Rcon app packaged for testing purposes atm.
  • UPDATED: CMake Changes New Warnings for detection of 64bit libraries (thx firefly2442), mainly for linux users that try to compile 64bit version.
  • FIXED: DB_Procedure now working
    IMPORTANT Procedures Support == limited, doesn't support Outputs anymore (with Poco 1.4)
    Was never working :(

edit: Poco 1.5 has Procedure Support for Outputs, but it was unstable i.e wouldn't compile extension on linux & caused crashes on Windows.

Will revisit at later data, DB_CUSTOM should be nearly as good if not easier to implement

Edited by Torndeco

Share this post


Link to post
Share on other sites

Keep up Torndeco :bounce3: You are doing great work here!

Share this post


Link to post
Share on other sites

Version 16

Just some small fixes...

  • FIXED: DB_CUSTOM_V2 issue when input had (sub-)string as $INPUT_x in it.
  • FIXED: MISC Protocol returning false instead of 0
  • MISC: Just some minor code cleanup..
  • FIXED: Updated CMake Build on github.... was missing db_custom_v2.cpp (only effected people building from source)

Unless there is some bug-fixes, i will prob take a break for a small bit....

Note: if u have any issues with DB_CUSTOM_V2, please report them.

You can downgrade to V16 until i get issue fixed. DB_CUSTOM_V2 Code was pretty much rewritten.

Also all Queries relating to Atlis Life RPG please ask @ http://www.altisliferpg.com/ thanks

Edited by Torndeco

Share this post


Link to post
Share on other sites

@ Torndeco

Do you use it or plan to use it for your own project(s)?

Share this post


Link to post
Share on other sites

@ .kju

My original plans were to rewrite server code for 2017 / epoch to use extDB.

So i could run the server on linux without wine, i was assuming they would prob use an extension that was windows only.....

I kinda like the persistent gameplay they bring...

------------------------------

------------------------------

As for my own project(s)

I have barely started to code my own mission, a version of Wasteland / KOFH coded from scratch.

Love to add civilian AI to the warzone, less max players (see if server can handle it, but i am prob dreaming here).

Would need to code a linux version of ASM & a GUI App for it, to see how code effects performance etc :(

All i got done atm for the mission (its not much)

Loadout / Preview of Character & spawning(/respawning) in a AI heli overhead are done.

Map markers & triggers for capturing areas...

But its hard to get motivation to-do the rest of code atm

Main thing that puts me off is doing the UI work / finding Images in a Open License i can use....

------------------------------

------------------------------

I know its not an original idea.... but i really need a something to put on a server.

At the very least it will allow me to test some new rcon code for win / linux servers...

If you are thinking of some new project, give me a shout

btw congrats on

http://forums.bistudio.com/showthread.php?182072-All-in-Arma-Terrain-Pack-%28AiA-TP%29-A1-A2-OA-terrains-from-BI-in-A3

To late for me to mess with it tonight but gonna check it out tomorrow :bounce3:

Edited by Torndeco

Share this post


Link to post
Share on other sites

Hi Torndec

Great work, just a small question. As I'm a total noob when it comes to DB and before I'll start busting my a@# learning how to use it...

But can this give me the ability to have a main cluster DB, what I mean if I have several servers running ArmA will they be able to save data to one DB, even if they are not on the same machine?

Share this post


Link to post
Share on other sites

@shay_gman

Yeah the servers can use the same Database i.e using MySQL to store infomation....

The Database Servers can be remote ones aswell.

Also if you don't have a static ip u can put in a normal address (FQDN) into the IP settings for extdb-conf.ini to connect to.

Recommend u use MySQL and not SQLite, since SQLite locks the file when updating etc....

Might become a perf issue if several servers updating the same SQLite DB often.

Edited by Torndeco

Share this post


Link to post
Share on other sites

@Torndeco

Thanks mate. Although most of what you have wrote is still Chinese to me I'll look into this.

Share this post


Link to post
Share on other sites

Sorry for little offtopic incursion :o

@ Torndeco

Sorry for the late response!

Very interesting to hear your motivation and interest - is the persistence your main topic,

or is it also the the hunter-gather aspect of DayZ gameplay?

What are the main elements you like in Wasteland and KOTH that made you considering write your own?

Share this post


Link to post
Share on other sites

@.kju

Persistence is nice, it means u can play for 30mins come back later + continue on.

Otherwise you would be like do i bother to play for only 30mins, and start fresh again later...

Also means public players tend to stick to server + build up a regular player-base... for those not in a gaming community / clan.

DayZ i like the potential it had, like DayZ Standalone when it first came out (haven't bother to play in over 6 months) loot was very rare...

It added to the aspect of having to scavenge for loot, not wasting ammo etc...

Just development is to slow + lack of ability to mod it in anyway :(

Wasteland on Zargabad (was a-few servers with it for arma2 at some point) was fun for random fighting with friends....

Like the CQC fighting, no running around getting sniped by someone sitting near top of a hill.

KOFH is just the above except Sa-Mantra has done a great job, especially with UI work.

Also don't like Wasteland static missions seems like repetitive / boring.

Had a simple mission for Epoch Server back in the day, randomize location / loot / randomized groups of AI's different range from mission aswell.

Plus had a heli that would S+D between mission sites aswell.

Just to keep players on there toes..

Not sure if i will bother with missions, need to see how it works out

Plus thinking about add a HC for civilian AI in the warzone, and players get punished for killing civilians...

Makes players not shoot everything that moves.... i.e is that truck driving a road (friendly / hostile / nuetral).

Its still early days though...

Or maybe i just change the backstory to a gang war in a city *shrugs*

But starting to look like i need to make my own ASM for linux so i can measure performance.

@Everyone else

Fixed rcon app, was using hardcoded filename value.

Anyway will just be a small update this weekend.

Planned Changes are

- Logging different directories for different server instances

- Logging ability to have different log-names

i.e so u could have <timestamp>.log and another one named hacker<timestamp>.log + debug<timestamp>.log

- Adding new filter option for DB_CUSTOM_V2 where u define different chars to strip out of inputs / outputs i.e "{}"

Don't worry none of chances will break anyone sqf code, will just be optional extras

Share this post


Link to post
Share on other sites

Right small update....

Got me a static build of extDB working :bounce3:

Means no more needing a chroot enviroment or compiling extDB for linux users.

You can still compile from source etc if u want, allows you to optimize for cpu arch.

Just got some minor issues with logging code to get sorted first.

Just got afew small things todo after way & should have it uploaded later today

Share this post


Link to post
Share on other sites

Update:

Linux Static Build of extDB @ https://github.com/Torndeco/extdb/tree/dev/_linux_release

Requirements

You just need to install libtbb2:i386

Debian / Ubuntu

apt-get install libtbb2:i386

Note for Debian Users

If you are compiling extDB, you will need to update your Boost Library.

This is because there was an issue with static linking + Poco Logging Library.

If this is to much hassle just grab the static build.

Differences between static / normal

Static is just easier to install for people new to linux + no need to compile / chroot environment.

Normal version is optimized for your cpu, this may make very difference. Since most of work is done via libraries.

Share this post


Link to post
Share on other sites

Version 17

Just afew minor changes / fixes..

Main thing of note is linux static builds :bounce3:

  • ADDED: Linux Static Build of extDB @ https://github.com/Torndeco/extdb/tree/stable/release/linux
  • FIXED: Compress MySQL Option
  • FIXED: DB_CUSTOM_V2 template fix exists check
  • FIXED: extDB exception error on arma3server shutdown on linux
  • UPDATED: Windows Builds using Boost 1.56 + latest Intel tbb43_20140724oss_win
  • MISC: Logging Code Changed back to Boost Log (done for Linux Static builds)
  • MISC: Added Thread ID to logs info, makes it easier to debug log information involving multiple threads working at once.
  • MISC: Misc Fixes for DB_BASIC

Windows Users:

Windows Builds, are built using newer 1.56 Boost & Intel TBB.

Please copy over the latest tbb / tbb_malloc dlls as they have changed.

Debian Users need newer Boost Libraries if they are compiling from source.

Otherwise just use the static builds.

Edited by Torndeco

Share this post


Link to post
Share on other sites
Guest

Release frontpaged on the Armaholic homepage.

==================================================

You are not registered on Armaholic, or at least not that we are aware of. In the future we offer the possibility to authors to maintain their own pages.

If you wish to be able to do this as well please register on Armaholic and let me know about it.

This is not mandatory at all! Only if you wish to control your own content you are welcome to join, otherwise we will continue to follow your work like we have always done ;)

When you have any questions already feel free to PM or email me!

Share this post


Link to post
Share on other sites

We've started running this on Arma 2 Island life, works beautifully. Thank you again Torndeco, amazing work.

Share this post


Link to post
Share on other sites

Hi,

So glad I came across this. Have no big love for the HiveExt.dll at the moment.

Do you have information available for which queries are run against which tables for the basic_v2 commands please.

Share this post


Link to post
Share on other sites

Please sign in to comment

You will be able to leave a comment after signing in



Sign In Now

×