Rudder

Since commit:b8362c25 , I limit the time and size of request (to 10s and 1000 entries).

The solution is not ideal because for now, if a limit is reached, the user just don't know anything about it.

So, we have to make a big red alarm to explain it to the user, and to say him that he should be more accurate in its search.

That leads to two other questions:
- what should be the behaviour with dyn group on such case (at least the same) ?
- what to do for policies that should be applied to ALL the park (and say the park is 100 000 servers) ? Just forbid it and explain to the admin that he doesn't really have 100 000 identical servers ?

François ARMAND a écrit:

Since commit:b8362c25 , I limit the time and size of request (to 10s and 1000 entries).

The solution is not ideal because for now, if a limit is reached, the user just don't know anything about it.

So, we have to make a big red alarm to explain it to the user, and to say him that he should be more accurate in its search.

That leads to two other questions:
- what should be the behaviour with dyn group on such case (at least the same) ?

I think we can have dyn groups of > 1000 servers, but only display the first 1000 to the user in the GUI. Even over 100, it seems unlikely the user will actually look at all of them on the web page. In this case, we could restrict getting the full list to specific operations like : exporting the policies to promises-writer, and a special link to download the full list (in CSV format, for example). What do you think?

- what to do for policies that should be applied to ALL the park (and say the park is 100 000 servers) ? Just forbid it and explain to the admin that he doesn't really have 100 000 identical servers ?

If a client has 100 000 servers, I'm sure he can pay for 2 more powerful servers to be dedicated LDAP CMDB servers, and we'll get much better throughput/performance on search from our CMDB. Don't forget our current server is really undersized for the size of it's DB : it's a small VM with 1 GB of RAM.

Also, getting the full list of servers should be fast. I think it's just searching on various criterion that may be slow.

Jonathan Clarke a écrit:

I think we can have dyn groups of > 1000 servers, but only display the first 1000 to the user in the GUI. Even over 100, it seems unlikely the user will actually look at all of them on the web page. In this case, we could restrict getting the full list to specific operations like : exporting the policies to promises-writer, and a special link to download the full list (in CSV format, for example). What do you think?

Yep, seems the adequate things to do. All in all, it's the path I'm walking on.

If a client has 100 000 servers, I'm sure he can pay for 2 more powerful servers to be dedicated LDAP CMDB servers, and we'll get much better throughput/performance on search from our CMDB. Don't forget our current server is really undersized for the size of it's DB : it's a small VM with 1 GB of RAM.

I've that in mind, no problem :)

Also, getting the full list of servers should be fast. I think it's just searching on various criterion that may be slow.

Yes, that is fast. Each query on it's own is rather fast on the server.
But what becomes non negligible is all the data encoding, network processing, etc. For 10 000 entries, we deal with something like 1Mo of data, and so a delay of several seconds just for the client processing (getting set of server 1, quering ldap with it, getting set of server 2, etc) - clearly not compatible with UI synchronous need.

I'm going to see how to do that as soon as the first policy will be link in the UI.

François ARMAND a écrit:

But what becomes non negligible is all the data encoding, network processing, etc. For 10 000 entries, we deal with something like 1Mo of data, and so a delay of several seconds just for the client processing (getting set of server 1, quering ldap with it, getting set of server 2, etc) - clearly not compatible with UI synchronous need.

It would be interesting to run the web server on the same machine as the LDAP server, to see how performance is impacted.

A lot less network latency, of course. I participated in some tests, which showed that using a WUI above a LDAP server was a good application to have the two programs on the same machine: basically because when the WUI asks the LDAP server for something, to WUI doesn't do any processing, it just waits while the LDAP server processes it's request. So they are not parallel applications.

How could we do this? Do you have a WAR we could deploy on the server in the lab?

Jonathan Clarke a écrit:
....

How could we do this? Do you have a WAR we could deploy on the server in the lab?

We will :)
For now, there is no stable version (not even built snapshot), so the only solution is to build from sources.

This doesn't match how we want to do this and how we can do filtering.
So I'm closing this ticket. But there is still a problem somewhere to be solved for 5k hosts.