extDB (arma3 extension linux/windows)

[torndeco 128]

extDB3
extDB3 is an Arma3 Addon for connecting to Databases (currently only MariaDB/MySQL).
The main purpose for extDB3 is for persistent missions in Arma.
Note it will require some knowledge about SQF & SQL to use.
extDB3 is also designed to be flexible & secure at the same time.

Links
https://bitbucket.org/torndeco/extdb3/wiki/Home
https://bitbucket.org/torndeco/extdb3/wiki/Contents
https://bitbucket.org/torndeco/extdb3/downloads

extBD/2 
Are retired

Edited May 22, 2016 by torndeco

[i3luevein 10]

Me like

Features:

Multi-Part Messages (i.e if output > outputsize set by arma)

[xyberviri 1]

Does this work with stored procedures, when using the original dayzmod bliss hive the issue with call999, other than sql injection, was if you issues a stored procedures it would blow up with a 2014 error on the 2nd call, which is caused by bliss not checking to see if another record exists via the dll.

http://stackoverflow.com/questions/17115919/mysql-commands-out-of-sync-error-c-connector

http://stackoverflow.com/questions/6826139/connector-c-mysql-error-code-2014-sqlstate-hy000-and-commands-out-of-sync

http://stackoverflow.com/questions/12903481/2nd-call-to-stored-procedure-gives-error-2014

http://opendayz.net/threads/bliss-saving-newly-created-vehicles-to-the-db-almost.7341/

[zooloo75 834]

Did you have fun writing the multi-part returns? :D

[tonic-_- 53]

Hmm, will have to give this a whirl.

[Gudsawn 93]

What exactly is this? Sorry if I sound a bit noobish, but the description is a tad vague.

[tonic-_- 53]

So... The documentation is pretty vague but the way I understand it it should be for example fetching:

0:DB_RAW:SELECT * FROM database.tablename

or something like that but it returns "Error Unknown Protocol", is the DB_RAW protocol just not working or was the windows release compiled version not up-to-date with the current push? Even tried what the wiki said in the calls general info of 'RAW' and still Unknown protocol.

*Edit*

Found out about having to add them in but after adding via the SYSTEM call part it just locks up and crashes (Using test-app).

Edited June 30, 2014 by Tonic-_-

[torndeco 128]

As for Documentation yeah it sucks alot atm.

I need to write a basic quick-start tutorial with basic sqf commands.

Hopefully i get time to-do it tomorrow.

-------------------

-------------------

Anyway simple guide, hope this clears up some confusion

Basicly @ mission.

Before u do anything u have to initialize extdb extension.

9:DATABASE:Database2    
// This causes it load Database2 Settings in the config file, if u skip this. 
//             It will crash if it attempts any database access later.

9:ADD:DB_RAW:fucktorndeco    
// This loads DB_RAW Module & Names it fucktorndeco
//            Originally i was coding plugins, but left some code in. 
//            This way u could try and randomize the Module Names @ startup. If u are paranoid


9:LOCK
// This is paranoid option, prevents loading any more options or getting extdb version info etc.
//      Basicly it disables all 9: System Calls   (there is no unlock option)


<INSERT REST OF YOUR MISSION SQF CODE>


Now since extdb is setup & loaded your database options & modules u wanted.
U can call

0:fucktorndeco:SELECT * FROM tablename
//            Note the fucktorndeco name, since we named it that above.
//            It only supports multiple connections to single database atm.  
//                  I can look @ coding support for connecting to multiple databases if needed.

---------- Post added at 20:09 ---------- Previous post was at 19:53 ----------

@ xyberviri

Am using Poco for Database access.

Currently using the stable 1.4 Poco it doesn't support procedures.

But the dev branch does support procedures.

Already looked @ 1.5 changes... should require zero changes to use it.

Tomorrow i will look @ coding support for procedures.

Basicly i will just compile 2 versions of extdb, (same codebase just add #ifdef #endif etc)

One version using Poco 1.5 with procedure support.

One version using Poco 1.4 without procedure support.

I will look @ it tomorrow, like to avoid using dev version if i can ;)

Worst case i might just have 2 versions of extdb compiled against poco, one with procedure + one without.

@ everyone else

New version on github, fixed mysql / odbc database connection.

Edited August 12, 2014 by Torndeco
Update info about procedure support

[Guest]

Release frontpaged on the Armaholic homepage.

extDB v2

==================================================

You are not registered on Armaholic, or at least not that we are aware of. In the future we offer the possibility to authors to maintain their own pages.

If you wish to be able to do this as well please register on Armaholic and let me know about it.

This is not mandatory at all! Only if you wish to control your own content you are welcome to join, otherwise we will continue to follow your work like we have always done ;)

When you have any questions already feel free to PM or email me!

[tonic-_- 53]

Am on Skype name = torndeco if anyone needs a hand...

As for Documentation yeah it sucks alot atm.

I need to write a basic quick-start tutorial with basic sqf commands.

Hopefully i get time to-do it tomorrow.

-------------------

-------------------

Anyway simple guide, hope this clears up some confusion

Basicly @ mission.

Before u do anything u have to initialize extdb extension.

9:DATABASE:Database2    
// This causes it load Database2 Settings in the config file, if u skip this. 
//             It will crash if it attempts any database access later.

9:ADD:DB_RAW:fucktorndeco    
// This loads DB_RAW Module & Names it fucktorndeco
//            Originally i was coding plugins, but left some code in. 
//            This way u could try and randomize the Module Names @ startup. If u are paranoid


9:LOCK
// This is paranoid option, prevents loading any more options or getting extdb version info etc.
//      Basicly it disables all 9: System Calls   (there is no unlock option)


<INSERT REST OF YOUR MISSION SQF CODE>


Now since extdb is setup & loaded your database options & modules u wanted.
U can call

0:fucktorndeco:SELECT * FROM tablename
//            Note the fucktorndeco name, since we named it that above.
//            It only supports multiple connections to single database atm.  
//                  I can look @ coding support for connecting to multiple databases if needed.

---------- Post added at 20:09 ---------- Previous post was at 19:53 ----------

@ xyberviri

For Procedure Support am using Poco for Database access.

According to 1.5 docs (dev version), it supports procedures, no mention of it in 1.4 (stable) docs.

But there is no mention of it in 1.5 changelog of it beening added in.

I will look @ it tomorrow, like to avoid using dev version if i can ;)

Worst case i might just have 2 versions of extdb compiled against poco, one with procedure + one without.

@ everyone else

New version on github, fixed mysql / odbc database connection.

Ahh I was missing out on the 9: DATABASE part now it's all working thanks.

[infection 1]

I've been testing it and I'm more than pleased, very good work! Thank you!

[jacob88 10]

Wow nice work looking at this now, thinking about how I could integrate this into my AntiHack

[torndeco 128]

New version with fixed RAW SQL for multiple row return.

Also enabled auto-reconnect for MYSQL (don't believe ODBC supports it, could be wrong)

Small Update + Question

Will be changing over to Poco 1.5 for database access. (Currently using 1.4)

No issues encountered & minimal change in code i.e already done locally :)

This way can add in support for SQL functions + procedure support, and access to MySQL handle.

Anyone used Procedures / Functions before for Arma, whats roughly the most arguments used ?

Will be a hardcoded upper limit, until i can find a better way todo it

Edited July 1, 2014 by Torndeco

[infection 1]

New version with fixed RAW SQL for multiple row return.

Also enabled auto-reconnect for MYSQL (don't believe ODBC supports it, could be wrong)

Small Update + Question

Will be changing over to Poco 1.5 for database access. (Currently using 1.4)

No issues encountered & minimal change in code i.e already done locally :)

This way can add in support for SQL functions + procedure support, and access to MySQL handle.

Anyone used Procedures / Functions before for Arma, whats roughly the most arguments used ?

Will be a hardcoded upper limit, until i can find a better way todo it

Great! I can't wait to get procedure support.

[torndeco 128]

Uploading new version to Github atm.

Changes

• Now Using Poco 1.5 (old version == poco 1.4, incase there is any issues)
  
• DB_RAW supports Procedures & possible functions haven't tested yet .
  
• Logging Added (no config option yet, will add later today)
  
• 
  

Todo / wish list for Tomorrow

• Add config option for logging
  
• Finish DB_PROCEDURE (this way u can call Procedures without enable DB_RAW). Plus it will be less calls to extension
  
• Type up Tutorial to Build Extension in 32bit schroot for Ubuntu Server.
  

Questions

Be interested to know if times for db access are still similar with new build, if anyone got time to check.

Anyone know if Procedure Output Values are local to database session or am i just going insane.

Notes

Old version is the previous version Poco 1.4, just incase there any issues.

Edited July 3, 2014 by Torndeco
Windows Logging Build = working now

[Guest]

Release frontpaged on the Armaholic homepage.

extDB v5

==================================================

You are not registered on Armaholic, or at least not that we are aware of. In the future we offer the possibility to authors to maintain their own pages.

If you wish to be able to do this as well please register on Armaholic and let me know about it.

This is not mandatory at all! Only if you wish to control your own content you are welcome to join, otherwise we will continue to follow your work like we have always done ;)

When you have any questions already feel free to PM or email me!

[torndeco 128]

Right Poco 1.5 was a bust... causes crashes

Reverted back to Poco 1.4

Github repo now has 2 branchs, stable & dev.

Its pretty much self explanatory. (help avoid me trying something big again & it breaks)

Anyway Linux Build Instructions @ https://github.com/Torndeco/extdb/wiki/Install-Linux---Chroot-Guide-%28WIP%29

Instructions are for setting up a 32bit chroot & to compile extdb & a link for arma3 server install instructions.

I literally wrote this down, as i setup my chroot enviroment on a fresh ubuntu server install.

Logging now works, if u have issues please make a issue on github & list your OS & filesystem thanks.

Also afew Atlis Life RPG servers have beening trying out extDB

With very encouraging reports of performance over Arma2MySQL for Atlis Life.

http://www.altisliferpg.com/topic/3689-extdb-support-separate-build/

Edited July 5, 2014 by Torndeco

[.kju 3244]

sweet :)

[infection 1]

Right Poco 1.5 was a bust... causes crashes

Reverted back to Poco 1.4

Github repo now has 2 branchs, stable & dev.

Its pretty much self explanatory. (help avoid me trying something big again & it breaks)

Anyway Linux Build Instructions @ https://github.com/Torndeco/extdb/wiki/Install-Linux---Chroot-Guide-%28WIP%29

Instructions are for setting up a 32bit chroot & to compile extdb & a link for arma3 server install instructions.

I literally wrote this down, as i setup my chroot enviroment on a fresh ubuntu server install.

Logging now works, if u have issues please make a issue on github & list your OS & filesystem thanks.

Also afew Atlis Life RPG servers have beening trying out extDB

With very encouraging reports of performance over Arma2MySQL for Atlis Life.

http://www.altisliferpg.com/topic/3689-extdb-support-separate-build/

Does this mean we won't be seeing procedure support anytime soon?

or does it still work with the new-old build(poco 1.4)?

[torndeco 128]

Sorry for confusion Procedures work with Poco 1.4.

Its just not mentioned the documentation directly, just 1.5 had some fancy c++ code to go along with it.

Short Version there will be a test build tomorrow with Procedure Support.

In the meantime, u can call Procedures using RAW SQL, not ideal i know.

Edited July 6, 2014 by Torndeco
Outdated Procedure Syntax Removed. Updated syntax @ extdb github wiki

[torndeco 128]

Update

New Test Build under dev-broken branch @

https://github.com/Torndeco/extdb/tree/dev-broken/windows_release

Changes

• DB_PROCEDURE Added info @ https://github.com/Torndeco/extdb/wiki/Calls:-DB_PROCEDURE
  
• DB_RAW trace log now includes results from database queries
  
• Arma2 Support - new config option to randomize config-file name.
  

edit:-

Hotfix Version 9

Disabled Intel TBB till if fix memory leak using it.

Issue caused by threads de-allocating each other allocated memory

Feedback on DB_Procedure appreciated... don't mess around with Databases to much

Edited July 10, 2014 by Torndeco

[Kingwolf 10]

Fist many tx for the great extDB!

I´ve tested the extDB with a local MySQL DB but the returning data is not clear for me.

My TestTable with two rows of testdata:

id int(11)

a varchar(20)

b int(11)

data varchar(4096)

data2 text

I´ve tried to select some data with the TestApp:

Some Init:

-----------------

9:VERSION
extDB: 10
9:DATABASE:Database2
extdb: Database Type: MySQL
extdb: Database Session Pool Started
extDB: [1]
9:ADD:DB_RAW:42
extDB: [1]
9:LOCK
extDB: [1]

Problem:

-----------------

2:42:select id,a,b,data,data2 from testtable where 1
extDB: [2,"1"]
extDB: DEBUG INFO: select id,a,b,data,data2 from testtable where 1
extDB: DEBUG INFO: RESULT:[ [1, "aaa", 111, "DATA DATA DATA", DATA2 DATA2 DATA2], [2, "bbb", 222, "NEW NEW NEW", NEW2 NEW2 NEW2]]
5:1
extDB: [1,[ [1, "aaa", 111, "DATA DATA DATA", DATA2 DATA2 DATA2], [2, "bbb", 222, "NEW NEW NEW", NEW2 NEW2 NEW2]]]

The question is, why ist the VARCHAR data returned with " arround it and the TEXT data without?

(There are no " in my tabledata)

Is this a bug or feature ;o)

Edited July 10, 2014 by Kingwolf

[torndeco 128]

@ Kingwolf

Its actually a kinda a bug for Poco Library (1.4)

Basicly MySQL is reporting Text Fields as Blob Data Type :/

There is a small fix for it in Poco 1.5, i could rob and patch in fairly easy. Like 4 lines of code.

But that means anyone compiling the code would need to patch Poco 1.4 i.e linux users

If they forgot it wouldn't be obvious whats gone wrong.

-----------------

Now i think about it its kinda convenient for saving Arrays to Database etc...

What i will do over the weekend is add DB_RAW_NO_EXTRA_QUOTES which won't add the extra quotations to var data types.

That way u have a choice of using either DB_RAW or DB_RAW_NO_EXTRA_QUOTES.

Plus it won't break anyone's existing code

@Everyone Else

Small bit of a rocky week for extDB over crashing etc...

All solved now, just don't use large datatypes like Long Text in your database please.

New Version 10 is running smooth :cool:

Planned Changes are

• Finish CMake Build System
  
• ADD new Random Seed Number for Unique ID (its currently in test-build dev-branch, note its in the extension only)
  
• Combine with Randomized 9:ADD: DB_RAW:<insert random generated text / number>, will make things harder.
  
• Steam Queries i.e getFriends List
  

Edited July 10, 2014 by Torndeco

[Kingwolf 10]

Perfect!

[Guest]

Release frontpaged on the Armaholic homepage.

extDB v10

==================================================

You are not registered on Armaholic, or at least not that we are aware of. In the future we offer the possibility to authors to maintain their own pages.

If you wish to be able to do this as well please register on Armaholic and let me know about it.

This is not mandatory at all! Only if you wish to control your own content you are welcome to join, otherwise we will continue to follow your work like we have always done ;)

When you have any questions already feel free to PM or email me!