Wednesday, November 23, 2016

Developing a 2D Board game with Python / Pygame : Part 2

Hi all,

Please check Part 1 before continuing to this section
In the last section, we covered board generation, leaving the actual network connectivity creation to this section.

Part 2 : Connecting to the server

Step 1 : Configure the game server

The given server (included in the github link), is made in C# as a Visual Studio Solution. Thus to run it
1) Download and extract the source
2) Install Visual Studio
3) Open lk.ac.mrt.cse.pc11/Tanks_server.sln
4) Once done, run the solution to open up the UI and the CLI

NOTE: IF you need to access the server from another machine, you need to make it listen to an interface IP. For that open up App.config and change the IP Address to something you get from ipconfig command.

Now with some analysis of the code and structure of the game server, the following can be seen.

1) We need to initialize a socket connection to the game server on the port 6000
2) As soon as we initialize and send the command "JOIN#" to game server, server will reply back to port 7000/
3) Thus, before sending the Join command we need to start a server on our client side that listens to port 7000.
4) Once that is initialized, our server continue to listen to the commands from the game server, through which various updates are taken.


Step 2 : Initialize socket connection

To ease everyone's life, I came up with a ServerClient class that handles all the things mentioned above. So in order to connect, all you need is to import the class and call the methods accordingly.
First save the socketclient.py which contains the ServerClient .

Now we use this class to connect to the server,

Note: Here the IP I have added is an external one since my server is running on a different machine. If you have both your client and Game server in the same machine, use 127.0.0.1 instead.

Save the below code as the new tank.py and run it. (Make sure the game server is running first). If all goes okay, you will see a string in the form "I:P:0...." printing on your console output. And the game server CLI output will show some text that says you have connected.


Step 3 : Drawing the board

Now that we can connect to the Game server and properly talk with it, next is to decode the board explanation given by the game server into graphical interface itself.

The format of the initialization code is
I:P0:<list of brick cordinates>:<list of stone wall cordinates>:<list of water cordinates>

So we simply do some splits and get cordinates of each element and draw them on our board. Since we have only implemented Brick walls in the sample I will show how to draw in the following code.

Once you run the tanks.py (remember that you need to close the game server and re run it before running your tanks.py, EVERYTIME), you will see that the Brick walls are shown on your interface in the same places as shown in the game server UI.

Okaaaay, so now you have the basic connectivity working, and I believe that you can continue building from here onwards. First make sure you add the logic to show all 4 different terrain types. Then make new Sprites, Coin and Lifepack and add them the same way we have used before.

I will try to update some future posts on the development, like auto vanishing coins and animations, but I guess this would work for now.

Cyah all







Tuesday, November 22, 2016

Developing a 2D Board game with Python / Pygame : Part 1

Hi all,
Quick introduction is the game is called Tanks and more details about the game will follow later since right now is time critical.
But just to begin,
1) The board is a 20x20 grid
2) 5 players play at the same time
3) There are 4 different times of terrain - Ground, Brick wall, Water, Stone wall
5) Other sprites include coins and health packs
6) Multiplayer, it communicated through a central socket-server

What you need

1) python 2.7 installed
2) pygame library installed

pygame is available as a standalone installer for windows, for most unix releases you can get it through standard repos (package python-pygame)

What is pygame

A very small 2D game development engine. It just covers the basic graphics handling like dealing with the graphics drivers and all, but on the other hand it leaves us with a lot of flexibility in coding. Also the learning curve for the syntax is almost non-existing! If you know your python, this is a piece of cake. If not well still learning python is pretty easy :D

Part 1 : Board generation

All the codes will be available on https://github.com/ManZzup/Tanks

In this section I would quickly cover the basics of pygame, and how to generate a board from a given input. I said generate because we will only receive a set of instructions from our multiplayer server. So we will construct the game generation accordingly.

Step 1 : Introducing pygame

This program just opens up a window and print some images on to it. This is the fundamental of game creation. If we need to animate all we need is to extend this to draw the same images again and again in a very rapid motion.So if you know to draw an image and to clear it, you can basically do animations!

Save this as tanks.py and run (python tanks.py)
If all goes good (fingers crossed) you will see a black screen with your image on it.



Step 2 : Making board cell

Let's get down to the real thing. SO to start off as i mentioned there are 4 types of board cells:
1) Ground cell
2) Water cell
3) Brick wall cell
4) Stone wall cell

So in pygame, each of these are called Sprites (read about them at pygame docs). Basically we would create python classes that extend the pygame.sprite.Sprite class
Why-- because Sprites deal with alot of other things like collision detection, Groups bla bla we'll come to them later
We start off by making a Ground Cell

Make sure you put ALL YOUR image files as the SAME directory as the script
Now let's modify our tanks.py to see how our cell looks like

When you run with python tanks.py you will see a small brown square in a huge black screen. IT WORKS!
Now i assume that you did the same for other cells as well, so i won't be covering that.

Step 3 : Making the Board Class

Our board class is basically a data structure that would hold
1) A data grid (2D int array)
2) All the cell sprites for all 400 cells
3) Any other board related info

So th following Board class defines all these. Note the concept of Group (pygame.sprite.Group) used here. A Group is a set of Sprites, the advantage is that when we group similar Sprites, we can apply some functions like drawing, scaling, collision detection onto them as a group.
Saves us a lot of coding and also it is efficient.

What's the data grid
Since our server will send us the info about various units in a very complicated form, I decided that those should be stored in a very efficient manner. SO the option I selected was to use an integer array (2D int array) to define each cell of our board. So how we define all the elements we have is here

We use 0 to indicate a Ground cell
0    -   Ground cell
-1   -   Brick cell
-2   -   Stone cell
.....
10  -    Player 1 Direction North
11  -    Player 2 Direction East
12
13

20  -   Player 2 Direction North

and likewise

So by referring to the above chart, in our Board class, first we define a method that would get the positions of each elements, populated by the server and mention them in the grid.

And the second method, draw_board is to loop those elements and construct the board by fixing the relevant cells.

 <<code to be added, check github>>

Again I assumed that you complete the above chart in a way that suits you plus the Cell classes.

Step 4 : Check board generation

Finally we just need to check if our board generation works. First we create a board instance and then manually feed the location of some brick walls. See that we just send a list of positions, just as we get them from the game server. This way when we finally communicate with server, we have less work to do on our side.

Then we call the draw_board function finally the cells.draw() function
Here's where the Group came useful. If not for the cells Group, we would have to iterate each and every cell and blit it to the screen. But here Group takes care of it ^_^.

Check your code with the following modified tanks.py

<<again due to some error, please check the code at github>>

DONE! So the first step is done, now i would say around 15% is completed. (Given that you take the extra step and construct the other cells, and do MODIFICATIONS as necessary)



Next is to communicate with the server,coming in Part 2.

Cyah






Friday, August 12, 2016

Writing integration tests for Apache CXF REST API that supports XML and JSON

Hey folks,

This comes up as a part of my ongoing GSoC project. The task as described in an earlier post was simple. To convert a SOAP service to REST. Yet the individual issues that comes up along the way sums up to writing those separate blog posts about them.

So this is the last phase of the project that I'm currently carrying out, that is to write integration tests for the now available REST web service. Now have the following assumptions before we go through the procedure

1) The service is implemented with Apache CXF and JAX-RS
2) It can consume and produce both XML and JSON based requests and responses
3) Service is up and running and accessible at https://localhost:5000/rest
4) Service has a one service method called getResponse that accepts XML or JSON requests (in a predefined way) and outputs according to the headers given.

Issue

Issue now is that although we can easily write unit tests for any given piece of code, writing unit tests for the web service api is just pointless. We need to be able to test the service online it self whenever we make a change to see that everything works good (the whole point of testing). So what we need are integration tests.
Moreover since my request/response architecture supports XML and JSON I should be able to send corresponding requests and check if I am getting the desired responses. So let's dig into the steps.

Steps

1) Add all dependencies

We need 2 things for our code to work
i) Apache CXF
ii) TestNG (Or if you preper you can use JUnit as well)

Since my project is all maven based, I will show how to add the required dependencies using maven. If you use any other dependency management software, feel free to find the libraries appropriately.



2) Prepare the directory structure

Before going into code we need to structure our codes so that the test classes doesn't mess up with our source code. Also keep in mind that since we have to test XML/JSON requests/responses we need to store these code files somewhere and read them as well. Hardcoding them inside the test classes would be an ugly option. So we have 3 things to keep apart from each other
i) Source code
ii) Test classes
iii) XML/JSON codes

Here's how I managed to seperate those



There are 2 main directories under the src directory
i) main : contains source code and it's resources
ii) test : contains test classes and it's resources

Since XML/JSON codes are part of the test classes, they are put within the resources directory under test directory.

3) So now we are ready to start with the code. First make a single class named TestService which extend Assert class under the test directory. (Chose the package names appropriately)

4) First we write a simple method to check if the service is online, if it's offline we cannot do any integration test on it.



NOTE:
If your service is secure (https), then probably the java WebClient will need to verify the certificates before any request can be made. If such verification cannot be done, an exception would occur. The easiest way to overcome is to make up your own keystore and specify the path to the keystore or truststore from the code. You can easily google that step, anyhow I added the following to the constructor of TestService class to mention the location of the trustStore to the WebClient. (This corresponds only according to my own configuration)



5) Everything's ready to go but we need to make some mock XML/JSON requests and responses first.

request-getResponse-1.xml


response-getResponse-1.xml



Keep those files in the xml directory under the resources.
Assume that when we send the contents of request-getResponse-1.xml over to the getResponse service method, we receive the response with the content in response-getResponse-1.xml.


6) Next we need a method to read the content from those 2 files. Here's a neat method that does the job, note that the string replacement at the end makes sure that things work correctly when we have to do assertEqual at the desired and actual responses at the end.



7) Let's write our first test method now
You can change / add / remove the headers as necessary, and keep in mind that this suits to a service method that reads the body of the request and processes it. If you play around a bit you can find the way to send any type of request to a service.



8) Here's the full code when all put together, and now you test the code. And TA-DAA
All test cases passed!

If you get any test case failures check the outputs of response and webResponse to see what's going wrong. So happy testing!


Reference

https://cwiki.apache.org/confluence/display/CXF20DOC/JAXRS+Testing

Friday, August 5, 2016

[GSoC 2016] REST implementation for WSO2 IS Entitlement Service

Hi all,

This post is to document the process followed, decisions taken and implementations done in implementing the REST service for WSO2 Identity Server's Entitlement Service. This was done as a GSoC project. If you are nto familiar with the WSO2 Identity Server, please first refer to my previous post which would answer most your questions.

Abbreviations

IS                      -  Identity Server
XACML           -  eXtensible Access Control Markup Language
REST                -  Representational state transfer
JSON                -  JavaScript Object Notation

Requirements of the service

  1. Allow RESTful access to 5 service methods (Already implemented in SOAP service)
  2. Allow XML XACML requests to be processed and responses to be given
  3. Allow XML web requests to be processed and responses to be given
  4. Allow JSON XACML requests to be processed and responses to be given
  5. Allow JSON web requests to be processed and responses to be given
  6. Exceptions should be handled globally and necessary responses should be given in a centralized manner
  7. Comply fully with the REST profile for XACML 3.0
  8. Provide swagger definitions for extra methods that are not specified in the REST profile


Implementation Decisions


  1. REST Framework - Apache CXF
       Apache CXF is a framework that allows easy deploying of REST Services. The key reasons for using Apache CXF for this implementation are

  1. Apache CXF already used with other REST Services (ex: SCIM inbound provisioning service). So libraries are already bundled with IS
  2. JAXB provides inherent support for marshaling and unmarshaling Java Beans to and from XML
  3. Additional libraries can be used to easily support JSON
  4. Exception mappers allow centralized exception handling

  1. Jackson for JSON support

Jackson is a JAX-RS provider for JSON, and it's already used within IS.

  1. Implementing JSON profile for XACML 3.0 defined here
  1. Using exception mapper to centralize the exception handling of the service and providing customized error messages from the service.

Final Implementation Design

Figure 1




Rough Class Diagram on critical components

Figure 2



Steps

1) Branching from SCIM inbound provisioning service implementation

Since the the above project is already implemented and functioning within IS, a copy of that was used as a starting point for the REST implementation of Entitlement Service. The following classes and packages were derived as it is from the SCIM implementation

packages
i) auth
ii) impl

classes
i) AuthenticationFilter
ii) AbstractResource
iii) ClearThreadLocalInterpretor

2) Removing SCIM related dependencies

All SCIM related dependencies were removed from the project. Since Authentication is not centralized within IS (and will be in future), the following authenticators were hardcoded.

i) BasicAuth
ii) OAuth

3) Restructure project paths and doing necessary changes to cxf-servlet.xml and web.xml

4) Adding JSON marshaling and unmarshaling to and from Java Bean support using Jackon

Currently Jackson 2.5.4 is added as an external library to the project. And the Jackson Json Provider was registered as a provider for the service. For which the following lines we added in cxf-servlet.xml

5) Define a new DecisionResource class which would handle the service endpoint requests

DecisionResource will extend AbstractResource. Accordingly the necessary service methods were then added (Refer Figure 2).

6) For methods getDecision and getDecisionByAttributes Request/Response converters were needed

By default Balana engine does not posses the capabilities to evaluate JSON XACML requests nor provide JSON XACML response. Hence 2 new classes were added to provide support.

i) JSONRequestParser : Converts JSON XACML request to Balana ResponseCtx object
ii) JSONResponseWriter : Converts Balanda ResponseCtx object to JSON XACML Response

7) For the rest of the web service requests, necessary Request/Reponse models were defined. Such defined models were

i) getHome : HomeResponseModel

ii) getDecisionByAttributtes / getBooleanDecision: DecisionRequestModel

iii) getEntitledAttributes :EntitledAttributesRequestModel, EntitledAttributesResponseModel

iv) getAllEntitlements : AllEntitlementRequestModel, AllEntitlementResponseModel

8) These models allow better Request/Response structures from the service

9) Custom Exception Mapper (EntitlementExceptionMapper) was added

This act as an intermediate to all exceptions occurred during the service method executions as well as JAX binding executions. Exception will not be handled inside the service methods, but rather will be thrown out to be caught be the exception mapper. Hence through which the exception handling procedure can be centralized. Custom error codes and messages were introduced. (Will be stated in a latter section)

10) Finally, the webapp is created to a war file and deployed to the IS webapps directory

It can be accessible by https://localhost:9443/wso2-entitlement/entitlement/Decision?_wadl

11) Implementing Swagger for populating service methods

Swagger is a framework that easily provide a globally understandable representation to the RESTful APIs. Since our REST service contained methods that were not defined by the XACML REST profile, swagger need to be used to populate the methods definitions of those additional methods and of the overall API itself. Swagger product a json/yaml file that nicely describes the service methods, input parameters and return values to any client that consumes it.

12) Writing integration tests for the service using testNg and Jetty 

Once all done, the integration tests were written. Used jetty module that comes with Apache CXF to fire up a web client and send and retrieve responses from the service. The full procedure is at http://manzzup.blogspot.com/2016/08/writing-integration-tests-for-apache.html



Code

The complete code can be found at
https://github.com/ManZzup/identity-framework/tree/master/components/entitlement/org.wso2.carbon.identity.entitlement.endpoint

Github Commits
https://github.com/ManZzup/identity-framework/commits/master?author=manzzup

Public JIRA Ticket
https://wso2.org/jira/browse/IDENTITY-4985


References







Thursday, July 14, 2016

phpmyadmin not found on LAMP install in Ubuntu 16.04+

Hey folks,

Just another quick one. Recently I installed Ubuntu 16.04 and Ubuntu 16.10 both uses the shinny new php7.0. And some newer version of apache as well. And both times when I installed phpmyadmin, and go to http://127.0.0.1/phpmyadmin or http://127.0.0.1/phpmyadmin
it gives me a simple, annoying 404!

Well there's a lot of solutions for this in Q&A threads, but for me the only working solution is that, you need to Include the phpmyadmin configuration at the end of the apache2 configuration, so that apache knows that phpmyadmin exists. Here's how

1) Open up apache configurations

sudo nano /etc/apache2/apache2.conf

2) Add the following to the very end

Include /etc/phpmyadmin/apache.conf

3) Save and exist and then restart the apache server

sudo service apache2 restart

4) Browse http://127.0.0.1/phpmyadmin

WORKS GREAT!!

Saturday, May 21, 2016

Tomcat error when deploying web app : addFilter, addServlet NoSuchMethodException

Hey peeps,

Just a quick update on something i found today. I was hacking things with a tomcat server (wso2 IS). My need was to duplicate a webapp and make some changes as I like. But when I changed those 'stuff' in the webapp source I had to specifically mention some dependency versions (build tool was maven) rather than obtaining them from the parent pom.

And when I generated the war and deployed it in the server I got the following error



Solution

It turns out that the reason for this error has nothing to do with the actual content of the web.xml
But a dependency mismatch
That is some libraries in your WEB-INF\lib directory of the war file is conflicting with some libraries already loaded. So the fix was for me to delete the lib directory completely and deploy which worked!

But according to some stackoverflow answers, deleting just tomcat-catalina.jar and/or all tomcat-*.jars will fix it

Happy Deploying! :D

Monday, April 11, 2016

Laravel 5 : Getting started for collaborative development

Hey folks,

Laravel, for those who aren't familiar is a php framework that is widely gaining popularity these days. This came into my attention when I had to implement an HR system in PHP. Usually I would have gone with Symfony, which I know has a larger support base. But the issue is that Symfony has a huge learning curve and I have to work with a team. Laravel on the hand has a less difficult learning procedure and it can be used to start implementation rapidly.

But Laravel it self is really confusing sometimes. Mostly if you are not familiar to tools like composer and git (for collaborative development). So here's a quick tutorial in which I mention steps I took to setup Laravel and a git repo for development.

Requirements

1) Git installed 
You can download git for any platform here,
https://git-scm.com/downloads
Make sure if in windows, that you add Git to shell commands when they ask to, so that you can access git from command prompt at any place.

2) LAMP / WAMP installed
Linux Apache MySQL PHP (LAMP)
Windows Apache MySQL PHP (WAMP)
You can do this per your liking

3) phpMyAdmin installed
If you are a windows user using popular WAMP like XAMPP, you will have phpMyAdmin already installed. LAMP users need to add it manually.

4) Composer installed
Composer is a dependency management tool for Laravel. Much like Maven  (if you are familiar) it will take a list of necessary dependencies and download these to the computer which removes the need of uploading all the 3rd party libraries everywhere.
Download and install composer according to the instructions given here
https://getcomposer.org/download/


Oookay now we are officially ready to start off with the things.

Steps

1) Make an online repository to host your source code
For collaborative development, first thing you need it to host your code somewhere. Since we are using git for source control, we need a host that supports git.
Github : will let you have public repos for free
Bitbucket : will let you have private repos for free (plus team integration with limits)

I went with bitbucket since our code is SUPER SECRET!
So goto bitbucket.org, make a new repo. let's say "AwesomeHR"
Now ask all your team members to sign up with bitbucket as well and in your repo settings, add all of the team members and give them write / read access.

2) Clone your repo to the local drive
It's time to get your repo to your hard drive. Goto your webroot directory (usually www, htdocs, etc depends according to your server) and clone the repo using this command,


git clone https://manzzup@bitbucket.org/manzzup/awesomehr.git


3) Installing Laravel

There are 2 ways of installing Laravel,
One is the official guide in their installation page using composer and laravel installer
https://laravel.com/docs/5.2

Second one is the hard code way in which we download the Laravel master branch from their github repository and extract it where we want. I had more success with this method in several platforms than using the installer (which has varying steps depending on the platform)

i. First download the Laravel master repo as a zip file from their Github repo
You can also use the following link for a direct download
https://github.com/laravel/laravel/archive/master.zip

ii. Next extract it to your newly cloned directory so that when you go into your AwesomeHR folder, it shown all the php files (not laravel-master folder, in that case move everything inside laravel-master to outside)


NOTE:
Now comes a very critical understanding. Laravel has 1000 different 3rd party libraries in use. Those libraries as well as the Laravel core source are NOT included in the zip file you just downloaded. why?

  • Because doing such would make it a very big file
  • We can't always have the latest version of the library if you pre bundle it with the Laravel source, any bug fixes won't reach once downloaded.

So here's how composer come into play. Composer's job is to download all those other libraries into the directory named "vendor" in your Laravel installation and setup them so that it works smoothly.
But keep in mind that you SHOULD NOT copy this vendor directory when you are sharing your code. In other words, the vendor directory be default has a .gitignore file which prevents ANY file inside that folder being uploaded to your remote repo.
Any collaborator using your code has to follow the steps of cloning your repo and then using composer to download the additional libraries.

iv. After that loong discussion, you use the following command while being inside of the extracted folder (AwesomeHR)

composer install

It will take some time for this to complete. Be patient my friend.

To check if all went good, goto http://localhost/AwesomeHR/public/ and you see this



Ta-Daaa you have a fresh Laravel installation now!


4) Configuring Laravel
Still the shit ain't working. Obviously you need to follow an installation procedure.
You need a database for Laravel to store all the data to start with.
Goto phpMyAdmin and make a new database called "laravel"
I will assume our database configuration is as follows

Database name : laravel
Database user : root
Database pass : root
Database host : 127.0.0.1

To make Laravel use these settings, open the .env.example file in your laravel root directory using your IDE or something like SublimeText. (Notepad won't work)
Edit the lines 5 - 8 to match your configuration

DB_HOST=127.0.0.1
DB_DATABASE=laravel
DB_USERNAME=root
DB_PASSWORD=root

Officially, you can start development with Laravel. You can try a few simple migration codes and make sure the installation is okay. Before you forget, you need to upload the code to the remote repo in bitbucket. Use the following command,

git add .
git commit -m "initial commit"
git push origin master

5) Collaborative development
To use the above code in a team mate's disk, follow these steps

i) Clone the bitbucket repo

git clone https://manzzup@bitbucket.org/manzzup/awesomehr.git

ii) Use composer to install the vendor files

composer install

iii) Do configurations as in step 4


Happy Laravel-ing folks!