Sitelet https://github.com/processone/tsung/pull/237
Skip to content

Add local_file_server - #237

Merged
nniclausse merged 1 commit into
processone:developfrom
tisba:local-file-server
Feb 26, 2023
Merged

Add local_file_server#237
nniclausse merged 1 commit into
processone:developfrom
tisba:local-file-server

Conversation

@tisba

@tisba tisba commented Apr 24, 2017 •
edited
Loading

Copy link
Copy Markdown
Collaborator

local_file_server is an alternative to the current file_server. It will distribute given files on start to all nodes and returns lines on request local to the requestor. This removes a severe bottleneck in tsung because there is now one gen_server per worker and per file. With file_server everything has to go though the controller which does not scale at all.

local_file_server works almost the same was the normal file_server. For obvious reasons (no global coordination), no global sequential access is possible.

Usage

local_file_server definition:

<options>
  <option name="local_file_server" id="users" value="/home/foo/.tsung/users.csv"/>
</options>

Accessing the local_file_server; Note that there is no support for order as sequential order cannot be guaranteed to the distributed nature of the local_file_server.

<setdynvars sourcetype="local_file" fileid="users" delimiter=";">
  <var name="username" />
  <var name="password" />
</setdynvars>

@tisba
tisba force-pushed the local-file-server branch from 56db0f8 to e04a776 Compare November 1, 2017 09:53
@tisba

tisba commented Nov 1, 2017

Copy link
Copy Markdown
Collaborator Author

Took a bit longer to finalise this change. Do you mind to take a look @nniclausse? I'm a bit unhappy about the very similar code for the distributed vs non-distributed initialisation, but maybe you have a good idea how to make this better.

@tisba
introducing local_file_server
6177ca7 
local_file_server is an alternative to the current file_server. It
will distribute given files on start to all nodes and returns lines
on request local to the requestor. This removes a tsung global
bottleneck because there is now one gen_server per worker and per
file.
@tisba
tisba force-pushed the local-file-server branch from e04a776 to 6177ca7 Compare November 1, 2017 15:45
@tisba tisba mentioned this pull request Nov 2, 2017
Open
@nniclausse

nniclausse commented Nov 20, 2017

Copy link
Copy Markdown
Contributor

Hi !
thanks, i'll try to review this PR soon.
Btw, i'm using git flow now, so the default branch is 'develop' and the master is only used for new releases. It should be easier to publish hotfix releases while new features are included in develop

@tisba
tisba changed the base branch from master to develop November 22, 2017 09:38
@tisba

tisba commented Nov 22, 2017

Copy link
Copy Markdown
Collaborator Author

Great!

I've changed the base branch to develop.

@rodo

rodo commented Nov 25, 2017

Copy link
Copy Markdown
Contributor

This is an awesome patch, hope to see it merged soon !

@tisba

tisba commented Feb 26, 2018

Copy link
Copy Markdown
Collaborator Author

Hey @nniclausse! Did you get a chance to take a look at this change set?

We've been using this for a while now, while it increases the start up time a bit and also memory usage, it has proven to work very well for for higher load scenarios when there is a lot of access to file servers.

@nniclausse

nniclausse commented Feb 26, 2018

Copy link
Copy Markdown
Contributor

Hi. Not yet unfortunately. Hopefully, i'll have some free time next week to do Pull Requests reviews and merge !

@tisba tisba mentioned this pull request Mar 28, 2018
Open
@tisba tisba mentioned this pull request Jul 11, 2018
Open
@nniclausse

nniclausse commented Jul 25, 2018

Copy link
Copy Markdown
Contributor

OK, i did a quick review and that seems OK for me. The only missing part is the documentation

@tisba

tisba commented Jul 26, 2018

Copy link
Copy Markdown
Collaborator Author

Okay, nice! I'll write something up so that we can get this out too.

@tisba tisba added this to the 1.8.0 milestone Jul 26, 2018
@tisba tisba self-assigned this Jul 26, 2018
@nniclausse
nniclausse merged commit 19c3322 into processone:develop Feb 26, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@tisba tisba

Labels

kind/feature

Projects

None yet

Milestone

1.8.0

Development

Successfully merging this pull request may close these issues.

3 participants

@tisba @nniclausse @rodo