Evaluation of Interregional Freight Generation Modelling Methods by using Nationwide Commodity Flow Survey Data

Email: wirach.hi@ku.th Abstract: A trip generation model is one of the four parts of the classical transport planning model, which explores the volume of trip or freight at the originating and destination points of a traffic analysis zone. The process of calibrating a trip generation model needs appropriate data. Freight transport data are always robust and a powerful calibration technique is required to handle the robustness of such data. The objective of this research is to evaluate the performance of the freight generation model, calibrated by the Artificial Neural Network (ANN), against the conventional linear regression model. The 2012 Thailand commodity flow survey data from National Statistics Organization of Thailand were used for calibration. Interprovincial freight shipment data, across the kingdom of Thailand (77 origins and 77 destinations), were divided into four categories-agricultural products, industrial products, consumer products and construction material. The results indicated that the regression based model failed to accord with the regression assumption, while ANN can also provide the same performance in explaining the relationship between dependent and independent variables. ANN is considered to be a better calibration technique as the concerned data do not accord with regression assumption.


Introduction
Freight flow plays an important role in transportation planning as well as passenger transportation and is primarily concerned with the economic activities of trip origin and destination. Freight flow data contain exhaustive information pertaining to many shippers, manufacturers and receivers and hence any dissimilarity in freight flow makes the collection of freight flow data complicated and costlier than the passenger flow data. Additionally, freight transport data are robust in comparison to passenger flow data.
The two freight data sources, which are commonly used to calibrate the transportation planning model include road side survey (Hirun and Sirisoponsilp, 2010;Kulpa, 2014) and Commodity Flow Survey (Celik, 2004;Park et al., 2012;Park and Hahn, 2015). The Commodity Flow Survey (CFS) collects shipment data from sampled shippers and the shipment data include information on Origin-Destination (O-D) of shipment, weight of shipment, value of shipment, etc. On the other hand, the roadside survey collects shipment data by interviewing drivers along the transportation link. The CFS may be preferable to roadside surveys for data accuracy. However, CFS surveys are costlier than roadside surveys, especially the surveys conducted on a national level.
With regards to the data sources in Thailand, it is observed that the kingdom faces intermittent supply of suitable data for transportation and logistics planning. The development of a comprehensive freight transportation data for strategic planning in Thailand is also at its early stages. The CFS began in Thailand in 2007 and was conducted for a second time in 2012. The 2012 CFS collected data from a large sample of shippers in Thailand, which was according to the Thailand Standard Industrial Classification.
Shippers with more than 16 workers composed the population of the survey. A total of 18,000 shippers were finally included in the survey. The CFS survey period was divided into four quarters. The office of the National Statistics of Thailand published the 2012 Commodity Flow Survey data for the public, as part of the research initiative that focussed on utilising commodity flow data for freight transport modelling.
The classical transport planning process, a four-step model, is widely used in passenger and freight transportation planning. The first step of this process is trip generation model, which explores the volume of a trip or freight at the originating and destination points of a traffic analysis zone. Trip generation may be calculated using different approaches and many previous researches have used the linear regression model for calculating trip generation (Tadi and Balbach, 1994;Novak et al., 2011;Naser et al., 2015;Yang, 2015;Patil and Sahu, 2015). However, it is observed that the regression model has a low performance and high prediction error and hence might not be a suitable transport model for calculating robust freight data.
The emerging software computing techniques have provided a new alternative method for dealing with transport modelling and this method is termed as the Artificial Neural Network (ANN). It is an artificial intelligence technique that mimics the function of the human brain. ANN can approximate a nonlinear relationship between the input and output variables of nonlinear and complex systems. ANN has been evaluated against many conventional computing methods in many disciplines (Moffat et al., 2010;Järvi et al., 2012), including transportation planning (Black, 1995;Mozolin et al., 2000;Celik, 2004;Tillema et al., 2006;Goel and Sinha, 2008;Arliansyah and Hartono, 2015). With regards to the trip generation modelling research, Al-Deek (2001) found that the Back-Propagation neural Network (BPN) model is more accurate than the regression model for predicting the levels of cargo truck traffic moving inbound and outbound at seaports. Employing truck trip diary data and roadside interviews data, Kulpa (2014) inferred that ANN is a better modelling tool than the multiple regression tool. The research suggested that ANN is suitable for trip generation modelling.
The objective of this research is to evaluate the performance of the freight generation model calibrated by the ANN against conventional linear regression model. The 2012 Thailand CFS data from the National Statistics Organization of Thailand were used for calibrating the freight generation model.

Data Description
The 2012 Thailand CFS collected comprehensive freight transportation data from the origin to destination. The CFS collected the following details: Number of shipments within one week, value and weight of shipment, shipment type, origin and destination of shipment, mode of transportation and import and export data. The 2012 Thailand CFS captured a total of 313,905 freight shipment records from the past four surveyquarters and the data categories covered agricultural, industrial, consumer and construction-based commodities. The data based on these commodities were subtracted for analysis in this research. The volumes of agricultural product, industrial product, consumer product and construction material are 145.130, 42.160, 18.115 and 171.764 million tons respectively. According to the definition of origin and destination, the maximum number of origins and destinations of freight transport in the province equals to 77. However, the CFS reported zero volume at some origin for all types of commodities and this zero volume might be outside the scope of CFS or a true zero. Unfortunately, matters outside the scope of CFS remained unaccounted for during the time of the study and hence the origin and destination with zero volume were excluded from the analysis. The details of origin and destination are shown in Table 1.

Development of Regression Model
The regression analysis was employed for calibrating freight production and freight attraction model. A total of four production and four attraction models were calibrated for all types of freight. The performances of these regression-based models will subsequently be evaluated against the model that is developed through ANN. The predictive equation from the regression model is as follows: Where: Y = The production and attraction of freight trips (expressed in million tons) The explanatory variables are employed to explain the freight generated at origin and attracted at destination. Therefore, it is important to specify the possible explanatory variables in the first step. These explanatory variables were sourced from relevant literature and adopted to meet the requirements of available and reliable data in Thailand. These variables can easily be procured by searching the explanatory variable sources provided by the related government agencies. Although Thailand's government agencies readily share information stored under their control, but most of their data are outdated. Among a number of variables, the possible explanatory variables that can be procured from government agencies for the freight generation model are Gross Province Product, population (Novak et al., 2011), crop area and employment (Novak et al., 2011;Kulpa, 2014). The proportion of data for model calibration and model testing must be considered, as model calibration requires a small data set (63 data sets for industrial products). Patil and Sahu (2015) used 84% of 64 observations for calibrating and 16% for validation. Al-Deek (2001) used 57 data points (74%) for training and 20 data points (26%) for validating the model. This research used 80% of the data for calibrating model and 20% was used for testing.
It is important to satisfy the assumptions about the population that provides the data in order to derive reliable results through the linear regression analysis. Diagnostic procedures are employed to determine whether the assumptions of linear regression are satisfied for the given model. These assumptions are as follows:

Independence
Errors in the conditional distributions are correlated. The explanatory variables are independent of each other. The independence assumption was verified by the Durbin-Watson statistics.

Normality
Errors are normally distributed with zero mean and a constant standard deviation. The individual data points of Y (the response variable) for each of the explanatory variables are normally distributed about the line of means (regression line). The Kolmogorov-Smirnov statistics was used to verify the normality of error.

Homoscedasticity
Errors in the conditional distributions should have constant variance. The variance of the data points about the line of means should be the same for each explanatory variable.
The developed regression models were evaluated using R-squared and t-statistics. The R-squared of the regression is the fraction of the variation in the dependent variable that is accounted for (or predicted) by the independent variables. The coefficient of determination R-squared ranges in value from 0 to 1. A value of R-squared equal to 1 shows a perfect correlation in the sample, while the coefficient of determination of 0 implies that the regression equation has not been helpful in predicting a y value. The R-square is given by the following equation: Where: i y = The mean of the observed data y i = The observed data ˆi y = model-predicted value of flow

Development of Artificial Neural Network
A multilayer feed-forward neural network model was used to build the ANN model. Among all the learning algorithms, the resilient backpropagation algorithm was found to be the fastest and most consistent learning algorithm for training the network. This feature of the resilient backpropagation has also been discussed in many previous studies (Kişi and Uncuoğlu, 2005). The resilient backpropagation also offers the advantage of altering the size of the weights, which is useful for avoiding the overfitting phenomenon (Dengel et al., 2013). This is one of the primary reasons for employing the resilient back propagation learning algorithm to train the ANN model. Celik (2004) suggested that the model performance will increase by using theoretically relevant and statistically significant variables. Thus, significant input variables from the regression analysis were used as input variables for the ANN model to ensure that the input variables have strong correlation with the output.
The R statistical software is a well-known open source statistical software and it is widely used in data analysis. The neuralnet package in R program was used for model calibration. The neuralnet package has the ability to train multilayer perceptron in the context of regression analyses, using backpropagation, resilient backpropagation with or without weight backtracking and the modified globally convergent version algorithm. The neuralnet package uses supervised learning algorithms for training the network (Günther and Fritsch, 2010). The neuralnet package in the R program was used in many researches (Soni and Abdullahi, 2015;Dengel et al., 2013;Nevtipilova et al., 2014).
Traditionally, the data are split into three-way crossvalidation datasets, which comprise training, validation and testing subsets. However, the proportion of subsets is variable. Arliansyah and Hartono (2015) split 85 datasets for evaluating ANN into 50 cases (58.8%) for training, 24 cases (28.2%) for testing and 11 cases (12.9%) to hold-out sample. Kulpa (2014) used 30 data points (60%) for training, 10 data points (20%) for validation and 10 data points (20%) for testing. Rasouli and Nikraz (2013) used 90% of the data (400 input vectors) for training and 10% for testing. However, Dengel et al. (2013) suggested that the dataset can be split into training and testing subset. They evaluated the performance of the training model using data, which were collected from different sites. This research aimed to evaluate ANN against regression method and hence used 80% of the data for training model and 20% was used for testing.
The number of neurons that should be used for training the network must be determined in order to achieve enhanced network performance. Tillema et al. (2006) varied the number of hidden nodes between 1 and 20. Järvi et al. (2012) repeatedly applied varying number of neurons, between 3 to 15 neurons and 100 repetitions, for training the network. Dengel et al. (2013) applied 1-12 neurons and ran 25 repetitions for each network to determine the appropriate number of neurons that must be chosen for training the network. We varied the number of neurons from 1 to 20 and ran 25 repetitions for each network. The Sum of Squared Errors (SSE) was used to evaluate the performance of ANN.

Evaluation of Performance of the Model
Where: e i = The actual value from CFS p i = The predicted value by models N = The number of data points

Regression Model
A linear regression model for all types of freight was developed and was evaluated using R-squared and t-statistic as shown in Table 2. Among all the explanatory variables, as mentioned earlier, the Gross Province Product and population are the significant variables, while crop area and employment are not considered as significant variables.
The t statistics values of the developed equations suggest that the coefficient is useful in estimating the freight generation. Concerning the freight attraction model, only the agricultural product attraction model obtained a high R-squared (near to 0.90), while the other attraction model obtained more than 0.50 R-squared. The value of R-squared suggests that the model has a moderate margin for prediction error. The developed freight production model shows unsatisfied performance and a moderate R-squared (near to 0.64) is obtained only in the consumer product production model. This means that 64% of the freight produced is influenced by the explanatory variable, while 36% are explained by other factors. The other factors in the freight production model reveal very low R-squared (near to 0.25), which implies that the developed model cannot successfully be employed for predicting freight production.
The results of examining independence and normality are presented in Table 3. The Durbin-Watson statistics value is between 1.5 and 2.5. The results indicate that all the developed models satisfy independence assumptions. However, the Kolmogorov Smirnov statistics reveal that all the models do not accord with normality of error assumption at a 5% significance level, implying that the relation between freight generation and explanatory variables do not accord with linear relation form. This factor contributed towards the employment of logarithmic transformation, commonly applied to address a nonlinear relationship (Novak et al., 2011), for developing the regression model. Logarithmic transformation is considered beneficial as it always produces positive value, while linear relationship may produce negative value (Rongviriyapanich and Suppiyatrakul, 2011). The predictive equation from the logarithmic transformation regression model is as follows: Where: Y = The production and attraction of freight trips (expressed in million tons) β 0 = Constant β 1 = Coefficient of X 1 X 1 = Explanatory variable e = Random error The results of logarithmic transformation regression model are presented in Table 4 and the statistics parameters for examining regression model assumption is shown in Table 5.    The t values of the developed equations suggest that the coefficient is useful in estimating the freight generation at 0.05 significant levels. The Durbin-Watson statistics value of all equations is between 1.5 and 2.5, which indicates that all the developed models accord with independence assumptions. However, the Kolmogorov Smirnov statistics of some equations indicate that the equations are missing the normality assumption. The examination of heteroscedasticity is presented in Fig. 1. The results reveal that the residuals are more or less evenly spread in a random manner along the horizontal line. Thus, the assumption of homogeneous variance is also satisfied. The developed equation obtains the R-squared between 0.275 and 0.508, which is lower than the linear regression equation. The value of R-squared suggests that the model has a moderate margin for prediction error. However, the Rsquared of the developed model is slightly lower than previous research, which reported the R-squared between 0.33-0.63 (Novak et al., 2011). There are only five equations that met the regression assumptions: Agricultural product attraction, industrial product production, industrial product attraction, construction material production and construction material attraction. However, the R-squared of the five models achieve low values, which reveal the performance of the model.

Artificial Neural Network Model
The five models-agricultural product attraction (M1), industrial product production (M2), industrial product attraction (M3), construction material production (M4) and construction material attraction (M5)-were used to compare the ANN model and regression method. The significant variables from logarithmic transformation regression model were used. The data for ANN training was transformed to logarithmic form for overcoming negative values. The numbers of neurons for each model were varied between 1 and 20, running 25 repetitions for each network. The Sum of Squared Errors (SSE) and number of neurons are show in Fig. 2 and the best performance models are show in Table 6.     The models M1, M3, M4 and M5 achieve a slightly different sum of square error, while the M2 provides more SSE. The number of neural, which is more than 5 neural, provides a slightly different reduction in SSE.

Comparison of Regression and ANN
As stated earlier, 20% data were used to compare the performance of ANN against regression based model. The comparative performance results of the models are show in Table 7. The value of RMSE is slightly different for model M1 and M2, while the value of RMSE for M3 M4 and M5 shows that regression method gives better results than ANN. On the other hand, the MAPE reveals that the artificial neural networks analysis provides satisfactory results against regression method for model M2 and M3 while achieving slightly different value for models M1 M4 and M5.
The results give scope to determine the reasons behind the performance of ANN on data that do not accord with regression assumption. The reasons were determined through the development of three models: Agricultural product attraction (M6), consumer product production (M7) and consumer product attraction (M8) were developed. The results are show in Fig. 3

Discussion
The well-known method, linear regression, was employed to develop freight generation model for four types of freight. A total of eight developed models were used for establishing the relationship between socioeconomic variables and fright generation and a few developed models provided high R-square value (0.895). However, all models suffered from gaps in the regression assumption. Therefore, the developed model was invalid. The logarithmic transformation was applied to overcome the problem. A total of five developed models were accorded with regression assumption. However, the developed model obtained the R-squared between 0.275-0.508, suggesting that the model has moderate margin for prediction errors. The R-squared of developed model was slightly lower than previous research, which reported the R-squared between 0.33-0.63 (Novak et al., 2011). Therefore, this model is considered to be unsuitable for capturing freight data, which involves complex economic activities at the national level. The robustness of freight data normally affects the model's performance and hence regression analysis may not be suitable for handling such complex data, irrespective of its transformation to enhance nonlinear form. The ANN model was developed using the significant variables from regression method to ensure strong correlation of explanatory variables with the output. The number of hidden neurons varied from 6 to 10 and errors ranged from 35.820 to 112.489. The two performance measurement, RMSE and MAPE, revealed that ANN provided satisfactory results against regression method for some models. Additionally, the performance of ANN against regression method was slightly different for other models. The finding suggested that ANN is suitable in freight generation modelling, which accords with previous research (Kulpa, 2014).

Conclusion
The emerging software computing techniques have provided a new alternative method for dealing with transport modelling. Although, ANN cannot be used to formulate equations on the relationship between dependent and independent variables and ANN's usage of 'black box'. ANN has the potential to disclose the imperceptible relationship between the dependent and independent variables. ANN can also provide the same performance relationship to explain the relationship between dependent and independent variables based on the data available. Contrarily, regression incorporates too many assumptions in the model formulation (Al-Deek, 2001). Although, the best results of ANN model are usually achieved through trial and error, the recent learning algorithm and powerful computing accomplished the process within a short time. Moreover, widely used ANN software for building ANN network and a few programs are open source (R program, Python). The continued development of ANN will facilitate advanced training algorithms and reduce calculation time. With regards to the results of this research, ANN, against regression model, would serve as a better alternative method for handling the robust freight data as the ANN data does not accord with regression assumptions.