Setting CiviCRM Buildkit to use for running Unit Tests in extension development

In this blog I will describe what I done so setup my local environment to run CiviCRM Buildkit for running unit tests in extension development.

Locally I run Arch Linux so some commands might be different on your machine.

First I download CiviCRM Buildkit to a local directory:

# first install all the required packages:
[jaap ~]$ sudo pacman -S npm nodejs #Pacman is the acrh linux command for apt-get
[jaap ~]$ cd /home/jaap
[jaap ~]$ mkdir bin
[jaap ~]$ cd bin
[jaap ~]$ git clone https://github.com/civicrm/civicrm-buildkit.git
[jaap ~]$ cd civicrm-buildkit
# This will download all tools needed to run Civicrm Buildkit
[jaap ~]$ ./bin/civi-download-tools 

# In my case I had to install/update nodejs
[jaap ~]$ npm update
[jaap ~]$ npm install fs-extra

Now I am ready to run AMP Config

[jaap ~]$ cd bin/civicrm-buildkit
[jaap ~]$ amp config
Enter db_type: mysql_dsn
Enter dsn> mysql://root:pass@localhost:3306 #my mysql dsn 
Enter perm_type: [0] "None"
Enter hosts_type: none
Enter httpd_type: none

Now I need to update my httpd config:

[jaap ~]$ sudo vim /etc/httpd/conf/httpd.conf #this is my appache config
# At the following line at the bottom of the file
Include /home/jaap/.amp/apache.d/*.conf #change this to your personal directory
#Restart apache
[jaap ~]$ sudo systemctl stop httpd
[jaap ~]$ sudo systemctl start httpd
# I have linked the build directory to my /var/www directory
[jaap ~]$ cd bin/civicrm-buildkit/
[jaap ~]$ rm -rf build
[jaap ~]$ ln -s /var/www build

Now test whether AMP is working:

[jaap ~]$ cd bin/civicrm-buildkit
[jaap ~]$ amp test

When AMP test did run without any errors you should add your civicrm-buildkit directory to your path

[jaap ~]$ vim .bashrc
# At the bottom of the file add the following line:
export PATH="$PATH:/home/jaap/bin/civicrm-buildkit/bin"

Now log out and restart after restarting you are ready to run Buildkit to deploy a site.

[jaap ~]$ cd bin/civicrm-buildkit/build
# Create a new site a new site in my /var/www will automaticly be linked to name.local.com in your situation that could differ.
[jaap ~] civibuild create dmaster --type "drupal-clean" --civi-ver "master" --url "http://dmaster.local.com" --admin-pass "admin1234"

Create an extension with a unit test in dmaster:

[jaap ~]$ cd /home/jaap/bin/civicrm-buildkit/build/dmaster/sites/all/modules/civicrm/tools/extensions
[jaap ~]$ civix generate:module org.civicoop.yourmodule
[jaap ~]$ cd org.civicoop.yourmodule
[jaap ~]$ civix generate:test CRM_Yourmodule_TestClassForTestingSomethingUseful

# Now run your test
[jaap ~]$ phpunit4 tests/phpunit/CRM/Yourmodule/TestClassForTestingSomethingUser.php

Links

* https://github.com/civicrm/civicrm-buildkit/blob/master/doc/download-update.md
* https://buildkit.civicrm.org/#/welcome
* https://civicrm.org/blog/totten/developers-extension-testapalooza
* https://phpunit.de/manual/current/en/appendixes.assertions.html

Use the delay mechanism to set the activity date in CiviRules

In this blog I will explain how to use the delay mechanism in CiviRules to set date field. In this example we will set the activity date in the action Add Activity. So that we can schedule an activity which should occur at the 3rd tuesday of the month.

CiviRules already has a delay mechanism to delay actions to a point in the future. But not that will specifically delay the action and nothing will be visible till the date the action should be executed.

The code in this blog could be used to set a contribution date in the future, or to set an end date of a membership or any other date field.

I am going to add the delay mechanism to an action to use it to set a certain date. I am also going to add the delay mechanism to the form where the user can enter the parameters for an action.

The first step we are going to do is to add a field for the Activity Date Time. We will call this field activity_date.

buildQuickForm()

The first step is to add the neccessary code to the buildForm of our action.

// CRM/CivirulesActions/Activity/Form/Activity.php

class CRM_CivirulesActions_Activity_Form_Activity extends CRM_CivirulesActions_Form_Form {

  public function buildQuickForm() {
    ....
    $delayList = array('' => ts(' - Do not set an activity date - ')) + CRM_Civirules_Delay_Factory::getOptionList();
    $this->add('select', 'activity_date', ts('Set activity date'), $delayList);
    foreach(CRM_Civirules_Delay_Factory::getAllDelayClasses() as $delay_class) {
      $delay_class->addElements($this, 'activity_date');
    }
    $this->assign('delayClasses', CRM_Civirules_Delay_Factory::getAllDelayClasses());
  }

The code above first retrieves a list with all the classes which could handle delays. Then for each classes it will make sure that the elements are added by the form. Then the list with classes is assigned to the template.

The line “$delayClass->addElements($this, ‘activity_date’); will allow the delay classes to add elements to the form. The second parameter is the prefix for each element name on the form. This prefix is used through the rest of the code to retrieve the data.

The template

After the elements are added to the form we should tweak our template to include the elements.

// templates/CRM/CivirulesActions/Activity/Form/Activity.tpl
...
<h3>{$ruleActionHeader}</h3>
<div class="crm-block crm-form-block crm-civirule-rule_action-block-activity">
...
 
 <div class="crm-section">
   <div class="label">{$form.activity_date.label}</div>
   <div class="content">{$form.activity_date.html}</div>
   <div class="clear"></div>
 </div>
 {foreach from=$delayClasses item=delayClass}
   <div class="crm-section crm-activity_date-class" id="{$delayClass->getName()}">
   <div class="label"></div>
   <div class="content"><strong>{$delayClass->getDescription()}</strong></div>
   <div class="clear"></div>
   {include file=$delayClass->getTemplateFilename() delayPrefix='activity_date'}
   </div>
 {/foreach}

...
</div>
<div class="crm-submit-buttons">
 {include file="CRM/common/formButtons.tpl" location="bottom"}
</div>

{literal}
 <script type="text/javascript">
 cj(function() {
 cj('select#activity_date').change(triggerDelayChange);

 triggerDelayChange();
 });

 function triggerDelayChange() {
 cj('.crm-activity_date-class').css('display', 'none');
 var val = cj('#activity_date').val();
 if (val) {
 cj('#'+val).css('display', 'block');
 }
 }
 </script>
{/literal}

The template consists first of displaying the activity_date select. After that the template for each delayClass is included and the prefix for the field names is also passed to the template.

The second part of the template is javascript to hide all the elements and only show the needed elements which is determined by the selected value in the activity_date select.

You can now see your form in action. There is no validation and saving yet.

Validation of the form

After creating the form and displaying the elements in the template it is time to validate whether the form is valid when it is submitted. We need to add our own formRule which then calls the validate function in the selected delay option (which done by the user by selecting an option for the Activity Date).

// CRM/CivirulesActions/Activity/Form/Activity.php

...

public function addRules() {
 parent::addRules();
 $this->addFormRule(array(
 'CRM_CivirulesActions_Activity_Form_Activity',
 'validateActivityDate'
 ));
}

static function validateActivityDate($fields) {
 $errors = array();
 if (!empty($fields['activity_date'])) {
 $activityDateClass = CRM_Civirules_Delay_Factory::getDelayClassByName($fields['activity_date']);
 $activityDateClass->validate($fields, $errors, 'activity_date');
 }

 if (count($errors)) {
   return $errors;
 }

 return TRUE;
}
...

The code above first adds the validateActivityDate function to the form as a validation rule. The validation function itself will get the class for the selected delay option and then pass the actual validation to the selected delay class. This is done by the line $activityDateClass->validate($fields, $errors, ‘activity_date’);  The last parameter is the prefix for the field names.

Saving the selected the activity date options

Now it is time to make sure the selected options are being saved with the rule. We do that in the postProcess function of the form. How you want to store the selected date for the activity date is up to you but in this example we are storing the selected option by serializing the delayClass.

// CRM/CivirulesActions/Activity/Form/Activity.php

public function postProcess() {
  ...
  $data['activity_date'] = 'null';
 if (!empty($this->_submitValues['activity_date'])) {
 $scheduledDateClass = CRM_Civirules_Delay_Factory::getDelayClassByName($this->_submitValues['activity_date']);
 $scheduledDateClass->setValues($this->_submitValues, 'activity_date');
 $data['activity_date'] = serialize($scheduledDateClass);
 }
 ...
 $this->ruleAction->action_params = serialize($data);
 ...
}

Setting the defaults

When the user edits the civirule the saved options should appear as selected in the form. This is done by setting the default values in the form.

// CRM/CivirulesActions/Activity/Form/Activity.php

public function setDefaultValues() {
...

foreach(CRM_Civirules_Delay_Factory::getAllDelayClasses() as $delay_class) {
   $delay_class->setDefaultValues($defaultValues, 'activity_date');
 }
 $activityDateClass = unserialize($data['activity_date']);
 if ($activityDateClass) {
   $defaults['activity_date'] = get_class($activityDateClass);
   foreach($activityDateClass->getValues('activity_date') as $key => $val) {
     $defaults[$key] = $val;
   }
 }
 ...
return $defaults;
}

Show a user friendly description of the selected activity date

The code below will show to the user which activity date is selected.

// CRM/CivirulesActions/Activity/Add.php
class CRM_CivirulesActions_Activity_Add extends CRM_CivirulesActions_Generic_Api {
...

public function userFriendlyConditionParams() {
  $return = '';
  ...
  if (!empty($params['activity_date'])) {
    $delayClass = unserialize(($params['activity_date']));
    if ($delayClass instanceof CRM_Civirules_Delay_Delay) {
      $return .= '<br>'.ts('Activity date time').': '.$delayClass->getDelayExplanation();
    }
 }

 return $return;

When the action is executed the the parameter for activity_date_time

The last step consists of setting the activity_date_time parameter in the action for creating an activity.

// CRM/CivirulesActions/Activity/Add.php
class CRM_CivirulesActions_Activity_Add extends CRM_CivirulesActions_Generic_Api {
...

protected function alterApiParameters($params, CRM_Civirules_TriggerData_TriggerData $triggerData) {
 $action_params = $this->getActionParameters();
 ...
  if (!empty($action_params['activity_date'])) {
   $delayClass = unserialize(($action_params['activity_date_time']));
   if ($delayClass instanceof CRM_Civirules_Delay_Delay) {
     $activityDate = $delayClass->delayTo(new DateTime(), $triggerData);
    if ($activityDate instanceof DateTime) {
      $params['activity_date_time'] = $activityDate->format('Ymd His');
   }
  }
 }

 return $params;
}

That is all what was needed to add the delay mechanism to the Add Activity Action.

Installing Pentaho 6.1 Community Edition on Ubuntu

In this tutorial I will explain what I did to install Pentaho 6.1 business analytics platform on an Ubuntu server.

I will also explain what I did to install Pentaho Data Integration, Pentaho Schema Workbench and Pentaho Report Designer on an Ubuntu Desktop.

For this tutorial I wanted to use my CiviCRM database as the source for reporting and analytics and I will use a seperate mysql database for the data warehouse. The Civicrm database is called civicrm  and the data warehouse database is called datawarehouse

I will use JNDI to connect to both databases.

Pentaho Business Analytics Server (or BI-Server)

Before we start make sure you have an ubuntu server up and running.

First install mysql database server, java, and the zip utility.

On the server enter the following command:

# sudo apt-get install mysql-server zip openjdk-8-jre openjdk-8-jdk libmysql-java

This will install all the required prerequisites.

Now it is time to download the BI-Server. You can download it from http://sourceforge.net/projects/pentaho/files/Business%20Intelligence%20Server/6.1/biserver-ce-6.1.0.1-196.zip/download

After the file is downloaded unzip it.

# unzip biserver-ce-6.1.0.1-196.zip

After you have unzipped it it is time to setup a mysql database for de BI Server and to set the connection for the mysql server.

I have followed the guideline from this blog: https://interestingittips.wordpress.com/2011/05/05/complete-pentaho-installation-on-ubuntu-part-2/

Enter the following commands to install the mysql databases.

# mysql --user root -p < /your/path/to/biserver-ce/data/mysql5/create_repository_mysql.sql
# mysql --user root -p < /your/path/to/biserver-ce/data/mysql5/create_quartz_mysql.sql
# mysql --user root -p < /your/path/to/biserver-ce/data/mysql5/create_jcr_mysql.sql

Now it is time to configure the BI-server to use those databases.

Open /your/path/to/biserver-ce/tomcat/webapps/pentaho/META-INF/context.xml and change the lines to match your mysql user and password also change it to use the mysql JDBC driver.

<?xml version="1.0" encoding="UTF-8"?> <Context path="/pentaho" docbase="webapps/pentaho/">
    <Resource name="jdbc/Hibernate" auth="Container" type="javax.sql.DataSource"
        factory="org.apache.commons.dbcp.BasicDataSourceFactory" maxActive="20" maxIdle="5"
        maxWait="10000" username="hibuser" password="password"
        driverClassName="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost:3306/hibernate"
        validationQuery="select 1" /> 

    <Resource name="jdbc/Quartz" auth="Container" type="javax.sql.DataSource"
        factory="org.apache.commons.dbcp.BasicDataSourceFactory" maxActive="20" maxIdle="5"
        maxWait="10000" username="pentaho_user" password="password"
        driverClassName="com.mysql.jdbc.Driver" url="jdbc:mysql://localhost:3306/quartz"
        validationQuery="select 1"/>
</Context>

Make sure the hibernate component will use mysql. Do this by editing the file /your/path/to/biserver-ce/pentaho-solutions/system/hibernate/hibernate-settings.xml and change it to:

<config-file>system/hibernate/mysql5.hibernate.cfg.xml</config-file>

Edit the file /your/path/to/biserver-ce/pentaho-solutions/system/hibernate/mysql5.hibernate.cfg.xml

<!--  MySQL Configuration -->
<property name="connection.driver_class">com.mysql.jdbc.Driver</property>
<property name="connection.url">jdbc:mysql://localhost:3306/hibernate</property>
<property name="dialect">org.hibernate.dialect.MySQL5InnoDBDialect</property>
<property name="connection.username">hibuser</property>
<property name="connection.password">password</property>
<property name="connection.pool_size">10</property>
<property name="show_sql">false</property>
<property name="hibernate.jdbc.use_streams_for_binary">true</property>
<!-- replaces DefinitionVersionManager -->

Edit the file /your/path/to/biserver-ce/pentaho-solutions/system/applicationContext-spring-security-hibernate.properties

jdbc.driver=com.mysql.jdbc.Driver
jdbc.url=jdbc:mysql://localhost:3306/hibernate
jdbc.username=hibuser
jdbc.password=password
hibernate.dialect=org.hibernate.dialect.MySQL5Dialect

Edit the file /your/path/to/biserver-ce/pentaho-solutions/system/applicationContext-spring-security-jdbc.xml

<!--  This is only for MySQL. Please update this section for any other database you are using -->
<bean id="dataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="com.mysql.jdbc.Driver" />
<property name="url"
value="jdbc:mysql://localhost:3306/hibernate" />
<property name="username" value="hibuser" />
<property name="password" value="password" />
</bean>

Modify the JNDI file by editing /your/path/to/biserver-ce/pentaho-solutions/system/simple-jndi/jdbc.properties

Hibernate/type=javax.sql.DataSource
Hibernate/driver=com.mysql.jdbc.Driver
Hibernate/url=jdbc:mysql://localhost:3306/hibernate
Hibernate/user=hibuser
Hibernate/password=password
Quartz/type=javax.sql.DataSource
Quartz/driver=com.mysql.jdbc.Driver
Quartz/url=jdbc:mysql://localhost:3306/quartz
Quartz/user=pentaho_user
Quartz/password=password

Add the mysql connector to the lib folder by making a symlink:

# ln -s /usr/share/java/mysql-connector-java.jar /your/path/to/biserver-ce/tomcat/lib/mysql-connector-java.jar

Edit the file /your/path/to/biserver-ce/tomcat/webapps/pentaho/WEB-INF/web.xml to prevent HSQLDB to start. Remove or make a comment from the lines below.

....

<!-- [BEGIN HSQLDB DATABASES]
<context-param> <param-name>hsqldb-databases</param-name> <param-value>sampledata@../../data/hsqldb/sampledata,hibernate@../../data/hsqldb/hibernate,quartz@../../data/hsqldb/quartz</param-value> </context-param>
[END HSQLDB DATABASES] -->

...


<!-- [BEGIN HSQLDB STARTER]
<listener> <listener-class>org.pentaho.platform.web.http.context.HsqldbStartupListener</listener-class> </listener>
[END HSQLDB STARTER] -->

...

That is it you are now ready to start pentaho.

# /your/path/to/biserver-ce/start-pentaho.sh

You can now access pentaho in your browser at http://your.server.name:8080

Install PDI on Ubuntu Desktok

Install the Java Runtime environment

sudo apt-get install openjdk-8-jre

Install the Java Development Kit

sudo apt-get install openjdk-8-jdk

After that run the following commands to check whether the installtion is correct:

# which java
/usr/bin/java
# java -version
openjdk version "1.8.0_91" 
...

Download PDI from [[http://sourceforge.net/projects/pentaho/files/Data%20Integration/6.1/pdi-ce-6.1.0.1-196.zip/download]]
After downloading install unzip

# sudo apt-get install unzip

After installing unzip extract the zip

# unzip pdi-ce-6.1.0.1-196.zip
# cd data-integration
# ./spoon.sh

 Install JDBC Driver

See [[http://stackoverflow.com/a/32873938/3853493]] for more information on how to install the mysql JDBC connector

# sudo apt-get install libmysql-java

This should install a file called mysql-connector-java in /usr/share/java

Now we need to create a symlink to this file in the data integration folder.

# ln -s /usr/share/java/mysql-connector-java.jar /your/path/to/data-integration/lib/mysql-connector-java.jar

Configure PDI to use JNDI

Open the file /your/path/to/data-integration/simple-jndi/jdbc.properties in an editor and add the following lines (settings are copied from above).  In the file below we define two connections one to the civicrm database (which is a data source) and one to the datawarehouse.

civicrm/type=javax.sql.DataSource
civicrm/driver=com.mysql.jdbc.Driver
civicrm/user=user
civicrm/password=*
civicrm/url=jdbc:mysql://your.civicrm.server:3306/civicrm
datawarehouse/type=javax.sql.DataSource
datawarehouse/driver=com.mysql.jdbc.Driver
datawarehouse/user=pentaho
datawarehouse/password=*
datawarehouse/url=jdbc:mysql://your.pentaho.mysql.server:3306/datawarehouse

Install Pentaho Schema Workbench on Ubuntu Desktop

Download schema workbench from [[http://sourceforge.net/projects/mondrian/files/schema%20workbench/3.12.0/psw-ce-3.12.0.1-196.zip/download]]

Unzip the file

# unzip psw-ce-3.12.0.1-196.zip

Install the JDBC driver. See instructions from above for JDBC on Ubuntu for PDI.

# ln -s /usr/share/java/mysql-connector-java.jar /your/path/to/schema-workbench/drivers/mysql-connector-java.jar

Then start the workbench with

# /your.path/to/schema-workbench/workbench.sh

Install Pentaho Report Designer on Ubuntu Desktop

Download report designer from [[http://sourceforge.net/projects/pentaho/files/Report%20Designer/6.1/prd-ce-6.1.0.1-196.zip/download]]

Unzip the file

# unzip prd-ce-6.1.0.1-196.zip

Install the JDBC driver. See instructions from above for JDBC on Ubuntu for PDI.

# ln -s /usr/share/java/mysql-connector-java.jar /your/path/to/report-designer/lib/jdbc/mysql-connector-java.jar

Start report designer with the following command

# /your/path/to/report-designer/report-designer.sh

Setup JNDI for report designer

Open the file in /your/home/.pentaho/simple-jndi/default.properties and add the following lines:

civicrm/type=javax.sql.DataSource
civicrm/driver=com.mysql.jdbc.Driver
civicrm/user=user
civicrm/password=*
civicrm/url=jdbc:mysql://your.civicrm.server:3306/civicrm
datawarehouse/type=javax.sql.DataSource
datawarehouse/driver=com.mysql.jdbc.Driver
datewarehouse/user=pentaho
datawarehouse/password=*
datawarehouse/url=jdbc:mysql://your.pentaho.mysql.server:3306/datawarehouse

Geluk

DSC_0025

Afgelopen dagen was ik voor mijn werk in het mooie Engelse Peak District. Dat is een gebied welke intrek is bij wandelaars vanwege de mooie heuvels die er zijn. Nu bestond mijn werk eruit dat we met zo’n 20 man ons opsloten in een vakantiehuisje en ondertussen verbeteringen aan CiviCRM aanbrachten dat kon zijn door de documentatie bij te werken, dan wel bugs op te lossen, of een nieuwe module te ontwikkelen.

Lees verder Geluk

Howto setup a CiviCRM development environment for fixing bugs

The steps for proposing a fix consist first of setting up an environment for fixing bugs and proposing a Pull Request (PR).

There are two ways for setting up an environment for fixing bugs, one is a fully running CiviCRM development environment, which requires buildkit, composer etc… In this tutorial I will explain how you could fix a bug in the code base and return your fix back to the community.  Of course, you have to fully test your fix.

Lees verder Howto setup a CiviCRM development environment for fixing bugs

Tutorial: installing redmine on Arch linux

#Yesterday I tried and succesfully managed to install Redmine on an Arch Linux server. I found it difficult to install Redmine on Arch because the instructions of Redmine and the wiki page on the Arch Linux wiki wheren’t clear, or at least I struggled with following those instructions. That is why I have written this tutorial.

In this tutorial we will install Redmine, but also MariaDB as a database server (required by Redmine) and Apache as the webserver.

I assume you have Arch up and running and for the convenience I also assume you have installed Yaourt

Lees verder Tutorial: installing redmine on Arch linux