torndeco

Version 23 Finally got another release done, just a week late. Main Changes are the following New Protocol DB_CUSTOM_V5, uses cached Prepared Statements + Options for INPUTS + OPTION :cool: extdb-conf.ini + extDB/ directory can now be located in same directory as extension, fallback location is Arma3 Directory For example now you can have @extDB/extdb-conf.ini @extDB/extDB @extDB2/extdb-conf.ini @extDB2/extDB Other Changes Updated Poco Library to 1.4.7p1 + custom changes. Just some minor fixes, and Poco Library now has optimization turned on for Windows Builds Custom changes are in github, basicly done to allow caching of Prepared Statements per DB Session. (used for DB_CUSTOM_V5) Known Issues Test Applications require tbb + tbbmalloc + extdb-conf.ini + extDB/ in the same directory as executable. Note there was a Version 22 Test Build, but was on github. I just never got around to testing it. https://github.com/Torndeco/extdb/tree/stable/release/windows https://github.com/Torndeco/extdb/tree/stable/release/linux

Right i am gonna get some needed sleep Will upload custom Poco Library Code Changes + compile new extDB Windows + Linux versions later today. Just a week late :j: DB_CUSTOM_V5 uses prepared statements, required some changes to Poco Statements + SessionPool so i could cache the Statements per Session. Sample of new format of config file [insertAntihackLog] SQL1_1 = INSERT INTO AntihackLog (Time, ServerID, uid, guid, name, hackType, hackValue) SQL1_2 = VALUES (NOW(), ?, ?, ?, ?, ?, ?); Number of Inputs = 5 SQL1_INPUTS = 1, 2, 2-BEGUID, 3-STRING, 4-STRING, 5-STRING More examples of INPUTS / OUTPUTS SQL1_INPUTS = 1, 2, 2-BEGUID, 3-STRING, 4-STRING, 5-STRING SQL2_INPUTS = 2-STRING-NOCHECK, 1-BEGUID-STRING OUTPUTS = x-NOCHECK, x-CHECK, x-BEGUID, x-STRING, x, x-STRING-CHECK Yes you can combine different options or override sanitize checks per value if needed... I will look @ doing the same for strip options later on. Will still support DB_CUSTOM_V3 for atleast another month, so there is no rush to update. But its recommended -------------- Other news Upcoming Poco 1.6 is still bugged for Linux Versions so no updating for that atm :( Believe its related to https://github.com/pocoproject/poco/issues/479 But i haven't had time or been able to debug the problem yet. Will finally add configureable VAC Banning (using RCON Commands) support to extDB. Should be done by next week 90% done already for months now.

Just a small status update.. Actually back doing abit of coding etc :) Upcoming changes for new release this weekend are 1) Prepared SQL Statement Support (DB_CUSTOM_V5) This should prevent against SQL Injection Attacks & also be a performance boost for SQL Statements that are called more than once. This will requires some custom changes to Poco Library, will have seperate Github to show changes etc, for those that build from source. Mainly the Poco Data Library code changes are so we can keep cache of called Prepared SQL Statements for each Database Connection (since they are database connections are dynamic based on config & db usage). 2) Also adding the ability to define to add quotes etc, when retrieving values from Database. i.e no more needed to have quotes in Database Fields etc... 3) Update to Latest Poco... Main change of note is optimization on Windows Poco Library Builds is turned on /O2

Yeah the floating buildings are from using older version of AiA TP. If you are manual installing AiA TP, don't forget to get AiA TP 2014 10 18 Patch Love the work on the island :)

Just store the unique ID as a string... Since the unique ID is sent back to extension as a string, there is no need to use parseNumber on it. All you are doing is adding extra work for Server. By converting it into a number + back to a string when fetching the result ------ Unique ID starting number is a randomize when extension is loaded... Its just a security feature

@RimBlock Exactly you can have multiple custom DB_CUSTOM_V3, they just need to called different aliases Also you need to do this before calling 9:LOCK STRIP Strip the character(s) out. Will run SQL Statement. STRIP+LOG Log it in extDB logs. Strip the character(s) out. Will run SQL Statement STRIP+ERROR Log it in extDB logs. Return [0,"Error Bad Char Found"]. Will not run SQL Statement Main thing to consider with STRIP+ERROR is when you are saving Player Names in a SQL Query etc...

@MattLightfoot Technical they are in violation of your Game Content Usage Rules But seems more of a guideline, sure the real rules are buried in the EULA somewhere... http://www.bistudio.com/community/game-content-usage-rules What can I do with content made by other users? ---------------- Regardless BIS has a very good / talented modding + coding community. That adds alot of value to the game engine Sure some people may not like some of the mods / missions. But they do bring in more Players + money for BIS to fund further development on engine. I really never seen a engine get some may patches / additional features after a release for so long like the Arma2 Engine. ------- Personally its really abit much to expect people that release their modding work for free. When someone abuses this constantly + ignores the license. BIS just tells authors sorry nothing BIS can do its up to yourself to sort it out. While also at the same time telling the authors of work they are limited to how they can make money off their work. i.e purely donations only, no fund raising to name a town / monument on a map It really would prob help if BIS took more pro-active steps, in helping community coders / modelers etc. You guys are more than capable of getting EULA changed for engine / battleye to add a loophole for BIS for extreme situations like this. Even if its just to make example of someone before it gets outof control. Taking the legal route for something you do in your free time, especially if offender is in a different country is really going to cost abit. Imagine going legal route + finding out the other person is only 15 years old... how are you going to recoup legal costs assuming you can based on local laws. Especially for something when the author doesn't make any source of income from the the addon / mission in question. Maybe its time for BIS to look @ charging a small yearly fee so people have the ability to encrypt their own EPO's for public releases etc.

Version 21 Just some minor changes.. Main thing is Windows Builds are now built using VS2013.. If you have issues make sure you tried installing http://www.microsoft.com/en-us/download/details.aspx?id=40784 Also the tbb + tbb malloc dlls have been updated/changed. ADDED: STRIP+LOG / STRIP+ERROR Options for Bad Chars in DB_CUSTOM_V3. FIXED: Issue if you select NONE for action for Bad Chars Action for DB_CUSTOM_V3. FIXED: Issue with wrong timestamp for filename, been off by a hour should be fixed now (via newer Poco). REMOVED: ODBC Support Removed, can be re-added if someone needs it. <--- Windows Binarys are now built using VS2013 + Update Various Libraries i.e Poco Boost MySQL Client Library. Linux Binarys updated Boost / Poco etc...

Not yet.. been inactive abit with extDB lately. There is a new Poco Stable Release with support for building using VS2013 So planning / hoping to get a new build out this weekend.. Ideally if i can will look at Add - Report bad chars encountered - See about expanding DB_CUSTOM features - Backport Poco 1.5 support for MySQL DateTime Datatype - Update to latest Boost Library Windows builds using VS2013 instead of current build system using VS2010

That is very good news to hear at long last I would appreciate if you guys don't use extDB in your rewrite. It is currently GPLv3 so you are free to use the extension or fork it all you want. But i am already considering rewriting parts of it + possible license change in the future. Something i really would like to avoid, but at the same time the actions of A3L is something i can't stand for. Good luck with the rewrite

@machine6fd still working on my rewrite of wasteland abit... I have a wip on github tonight/tomorrow. If you understand the SQL Statements / Database side and its just talking to extDB that is confusing... Just give me a shout on Skype & will help you out etc @Everyone else I really shouldn't need to mention this, don't forget to use the following options in your server.cfg examples @ https://community.bistudio.com/wiki/server.cfg Its to prevent people from reading your mysql details etc... allowedLoadFileExtensions[] allowedPreprocessFileExtensions[] allowedHTMLLoadExtensions[]

How is this still going on.... Caidan defense just keep changing Donation wise First it was everyone was Donating -> Tonic Mission had Donating at one point -> Donating doesn't effect interviews to get Beta Access -> forgot to update website with perks You were blatantly ignoring Arma Content Rules Code wise Tonic code is opensource -> someone pointed out the non-commerical / no derivatives parts to the license -> blah Tonic stole our code so is ok to steal his code. You are ignoring a legal license, because you believe your code was stolen... Seriously prove it.. Tonic Code is on github not hard to link parts you believe are stolen etc (why show arma stolen code) Community are the people that make SQF Code, you show a segment of code, that may have come from a forum post that both Life missions might have used etc... All i have seen code wise, is Tonic showing GUI Code comparisons + he even mentions were the images came from + who gave permission for usage etc.. ----------------- Please stop acting like a kid with his hand in the cookie jar... Changing your defense / reasons for your actions is just not helping... Different Addon / Mod Authors are already telling you to stop using their Addons/Mods Honestly SQF Code is not hard, Arma Dialogs are a pain though :P Just either start from scratch or use your old mission as a base and start to update the code etc... Btw, congrats first time i ever regretted releasing something as open-source. If i had rights to pull your rights in using extDB i would do so in a heart-beat

Yeah sorry about that, normally i wait abit before i break something i.e different Protocol Version. But there were a couple bugs i fixed + its relatively quick changes to config file to make it work... If you run into any other issues give me a shout or send me a msg on Skype . Note you could prob reduce the amount of threads extDB is using (these are worker threads and most of the time will be doing nothing anyways) Edit your extdb-conf.ini and try change the Threads to 4->8. Should have no real impact on extDB performance, think i might change the max default value in next version to 4. edit: Also will add warning when it detects no SQL Statement in db_conf file, looks like i overlooked that

Looks like you have v19 of extDB It has a typo/bug in v19 https://github.com/Torndeco/extdb/tree/stable/release/windows/20%20Debug%20Logging Its loading DB_CUSTOM_V2 instead of DB_CUSTOM_V3 :( Also there are some new / nicer features in DB_CUSTOM_V3 in v20 Example of new db_conf file with the features https://github.com/Torndeco/ArmA3_Wasteland.Altis/blob/revive_beta/extdb/db_custom/a3wasteland.ini https://github.com/Torndeco/extdb/blob/stable/Changelog.txt

Linux server doesn't create a logfile. You need to pipe the console output (all that scrolling text) into a logfile

Thats were the regrex rule, comes in. Extension will only return parameters that match the regrex rule (hardcoded or defined in extension configfile). That way its impossible for SQF code or hacker to get info on an important commandline option. Anyway just a random idea i had... Anyway back to coding

Very nice project + looks like very easy from mission maker to implement aswell :) Personally i like you are getting the command line arguments. Think i have to implement something similar myself. Just random idea i had, Add ability to retrieve set command line parameters from SQF that follow a regrex rule. arma3server.exe ........... --missionVarInstance=1 That way mission maker can ask extension for all variable with pattern --missionVar* it returns Instance 1 Its a small feature, but it could be useful. Anyway just a random idea i had. Nice to see more linux extensions

Look @ extDB/logs there will be an error log entry there. More than likely it will be complaining about bad Database Settings or cant find config file.... in both scenarios extDB will kill the server by default.

Version 20 Plenty of code changes / fixes... Going to Split the List here into 2 Sections, since most of the changes are for DB_CUSTOM_V3. Noticed afew areas lacking, while i be working on added extDB support for A3Wasteland. General Changes FIXED: extDB will now use empty string "" for value when value in Database is Null. Before returned string wouldn't compiled. FIXED: mutex Lock issue for MISC:BEGUID FIXED: SegFault when Data Supplied to some Protocols was empty. REMOVED: DB_CUSTOM_V2, will take less than 5/10 minutes to update config file to new format / features. REMOVED: DB_BASIC_V2, don't know of anyone using this Protocol. Its now obsolete with Multi-SQL Support in DB_CUSTOM_V3. If anyone needs a hand, give me a shout and will help you update to DB_CUSTOM_V3 DB_CUSTOM_V3 Changes ADDED: DB_CUSTOM_V3 Mult-SQL Statement Support.. you can now define multiple SQL Statements to run in sequence after each other. Only result from last SQL Statement is returned. ADDED: DB_CUSTOM_V3 $INPUT_STRING_x which will add "" around input values. This is integrated with Sanitize Value Check. ADDED: DB_CUSTOM_V3 $INPUT_BEGUID_x which will convert Player UID -> BeGUID + wrap it with "". ADDED: DB_CUSTOM_V3 'Option String Datatype Check' To disable / enable adding quotation around results. Was old behaviour for DB_CUSTOM FIXED: DB_CUSTOM_V3 were you have more than 9 Inputs, Max Inputs is now 999 CHANGED: DB_CUSTOM_V3 Renamed some options look @ example-a3wasteland.ini. Example of new DB_CUSTOM_V3 https://github.com/Torndeco/ArmA3_Wasteland.Altis/blob/revive_beta/extdb/db_custom/a3wasteland.ini Notice the following call [insertPlayerInfo+Save] SQL1_1 = INSERT INTO PlayerInfo (UID, Name, BattlEyeGUID) SQL1_2 = VALUES ('$INPUT_2', '$INPUT_STRING_3', '$INPUT_BEGUID_2'); SQL2_1 = INSERT INTO PlayerSave SQL2_2 = (ServerID, PlayerUID, CreationDate, LastModified) SQL2_3 = VALUES ('$INPUT_1', '$INPUT_2', NOW(), NOW()); Number of Inputs = 3 "extDB" callExtension ("1:DB:insertPlayerInfo+Save:" + _server_id + _player_name + _player_guid) With the above example can run 2 sql statements with a single callExtension. Also Player GUID is converted to BEGUID automatically aswell. A3Wasteland version with extDB still WIP, but showing nice performance gains etc...

The strip code is only ran against the $INPUT_ values... But i will look at adding options to log / return error out if it encounters a bad character. Not sure if will make it into next version or not (would need to make / use a different function). Sanity Checks is code from DayZ Epoch Hive https://github.com/Torndeco/extdb/blob/stable/src/sanitize.cpp Basicly a-z A-Z 0-9 wrapping in quotes / [1,2] / "test" / ["omg",[1,2],0] / 12345 etc... Try out the sanitize app if your are curious if a value will pass. Afew more features coming to DB_CUSTOM_V3 in next release WIP Changelog @ https://github.com/Torndeco/extdb/blob/dev/Changelog.txt Should be a release sometime tomorrow with a test version of a3wasteland using DB_CUSTOM_V3

[1, [1,[[0]]]] [1, --------] = OK from extDB retrieving Result, other values include 0 = error / 3 = wait / 5 multi-part message --- [1, ----] = OK from Protocol, other values include 0 = error ------ [[0]]] = Result Basicly extDB can return error messages i.e MySQL Server crashed... or if you had another process using SQLite alot + it failed run SQL Statement due to file locking. Its possible to code SQF to handle errors if you want to, most people prob don't need todo this. But can be nice for debugging in SQF. Also the code for Protocols is abstracted / seperated from extDB main code, makes it easier to add / change / remove Protocols. [[0]] [[Row 0],[Row 1]] [[Row 0 Column 0, Column 1],[Row 1]] You have to remember SQL Query could return multi-row sql results. So you got 2d arrays, array of row results, containing array of column values. ------------------ You are using SYNC method to get results from Database. This will block arma engine until SQL Statement is complete = bad. I would only use SYNC method for Server Startup it avoids overhead of worker threads... even then i don't like it / recommend it. Use ASYNC method to get results from Database. This wont block arma engine as much, since work is done in a different thread. 1:<.....> ASYNC + no results returned i.e for Update / Insert etc 2:<.....> ASYNC + return results via unique_id i.e for Select Then you use 4:<Unique ID> / 5:<Unique ID> to get the results You can use Select *, you more than likely have a typo mistake... try run the command in a program like MySQL Workbench to make sure its correct. ------------------ Sanitize Check, only checks the values uses same code as DayZ Epoch Hive etc, returns an error if bad value (i.e won't run sql statement). Strip Chars will actually remove the offending chars and then do Sanitize Check (if enabled) and then it will run the sql statement.

Version 19 Haven't had motivation to-do much lately, so only afew small changes... Been meaning todo update for abit. ADDED: DB_Custom_V3 Protocol, config file is abit more cleaner. Multiple SQL_x lines when added together have space added between them i.e SQL_1 + " " + SQL_2 + " " + SQL_3 etc Ability to set Default Settings Strip List of Characters Option for Input Values ADDED: New Get Results Method. 4:<Unique ID> FIXED: Trace Log, showing input instead of result info in logs. MISC: Added warning if trying to use Trace Setting in normal extDB. 4:<Unique ID> Most of the time, you will be query Databases for small info like player stats / inventory etc... So the current method of calling 5:<Unique ID>. Until you get an empty string is abit overkill. + 1 extra callExtension for fetching results. You can now retrieve results by calling 4:<Unique ID>. If result fits into single message (i.e <= armaoutputsize), extDB will return [1,<result>]. + 0 extra callExtension. If result doesn't fit into a single message (i.e > armaoutputsize), extDB will return [5]. This means you need to use 5:<Unique ID> to fetch results. + 2 extra callExtension Best part with this change is it doesn't break any SQF code, you can choose to ignore it if you like. Or you could use both methods in your code, if you want to optimize usage abit. Getting Player Info could use 4:<Unique ID> (since most of the time it will fit into armaoutputsize) Getting All Vehicles could use 5:<Unique ID> (since it doesn't happen much + it will prob be greater than armaoutputsize).

Why cross post issue ??? I was answering your post in Altis Life Thread The last thing i told you to grab debug version of extDB + turn on trace in the logging options. At the very least it will help you out last call to extDB. Regardless someone else was helping you, according to their post. http://www.altisliferpg.com/topic/6282-setup-client-please-wait-problem/?p=41190 Your clients are never getting loaded into server, they are getting stuck inside. waitUntil {!isNull player && player == player}; https://community.bistudio.com/wiki/player https://github.com/TAWTonic/Altis-Life/blob/master/extDB-Build/Altis_Life.Altis/core/init.sqf#L14 Thats is an arma issue not extDB, maybe related to groups (i don't know off hand, since i never messed around with SQF Code + High Player Counts) Your clients are getting stuck well before server even asks extDB anything at all. ---------- Post added at 22:42 ---------- Previous post was at 22:34 ---------- Anyone using DB_CUSTOM_V2 ? Anything in particular you like to see get added in the next version. New DB_CUSTOM_V3 in v19. Planned Changes are [Default] Option Field to define default values i.e sanotizie check, number of inputs etc... Strip Chars = true Strip Chars List = '/\|;,{}- Also new option for stripping chars from input values. Saves you having todo this in SQF Code. Note if ":" char is in any input values (it is used as Protocol Divider), it will cause fail with error message So you may still want to strip this char or just add SQF Logic to handle this error message. [0,"Error Incorrect Number of Inputs"] Last thing is SQL_x will have space added between values. SQL = SQL_1 + " " + SQL_2 + " " + SQL_3 Instead of currently with DB_CUSTOM_V2 SQL = SQL_1 + SQL_2 + SQL_3

Version 18 Mainly just a bug fix + optimize return output release Also nearly forgot V1 Protocols got removed. MAJOR: Removed V1 Protocols, been over a month. <-------- ADDED: 9:OUTPUTSIZE debug command, returns outputsize for armaextension. Also logs this info to extdb logfile. ADDED: config option "Error Database Kill Server" to enable/disable killing server if Database Connection Fails. FIXED: Fixed (+ Tested) Template File Check for DB_CUSTOM_V2 FIXED: Killing Server if Database Config Section Missing. (If killing server == enabled see above) FIXED: Removed some arma2 code from compiling for linux builds. (Randomized Config Option) FIXED: Test App Input Limit: by MaHuJa https://github.com/MaHuJa MISC: Removed White-space Padding from Returned Results. MISC: Various small Logic Fixes to code + more error outputs i.e like if you try to 9:ADD:Database Twice etc... MISC: Misc Fixes for DB_BASIC Note: I still need to compile Linux Builds + Test Applications... Will be done in a hour or 2... Just gonna grab a coffee etc first.

@RimBlock Yeah stop trying to get results from Procedures its gonna be broken. Its an issue with the Poco Library i am using for Database Connection. Anyway try use v16 for DB_CUSTOM_V2 its working there. There is an issue with it checking if extdb-conf.ini exists instead of the sql template file, otherwise it should be good. Looks like i broke it in v17 when i fixed it :j: , swore i checked / tested it beforehand. Note: the tbb + tbbmalloc dll's are different for v16 + v17, so don't forget to replace them Am normally on Skype if u run into other issues etc Disclaimer: Atlis Life Admins should post on Atlis Life Forums, i look on the forums there thanks