Data Scientist

The overall parameters have been divided into 3 categories by XGBoost authors:

General Parameters: Guide the overall functioning
Booster Parameters: Guide the individual booster (tree/regression) at each step
Learning Task Parameters: Guide the optimization performed

The various steps to be performed for Parameter Tuning are:

1) Choose a relatively high learning rate. Generally a learning rate of 0.1 works but somewhere between 0.05 to 0.3 should work for different problems. Determine the optimum number of trees for this learning rate. XGBoost has a very useful function called as “cv” which performs cross-validation at each boosting iteration and thus returns the optimum number of trees required.

2) Tune tree-specific parameters ( max_depth, min_child_weight, gamma, subsample, colsample_bytree) for decided learning rate and number of trees. Note that we can choose different parameters to define a tree and I’ll take up an example here.

3) Tune regularization parameters (lambda, alpha) for xgboost which can help reduce model complexity and enhance performance.

4) Lower the learning rate and decide the optimal parameters .

1) Hyperparameters are certain values or weights that determine the learning process of an algorithm.

2) XGBoost provides large range of hyperparameters. We can leverage the maximum power of XGBoost by tuning its hyperparameters.

3) The most powerful ML algorithm like XGBoost is famous for picking up patterns and regularities in the data by automatically tuning thousands of learnable parameters.

4) In tree-based models, like XGBoost the learnable parameters are the choice of decision variables at each node.

5) XGBoost is a very powerful algorithm. So, it will have more design decisions and hence large hyperparameters. These are parameters specified by hand to the algo and fixed throughout a training phase.

Generally, the XGBoost hyperparameters have been divided into 4 categories. They are as follows -

1) General parameters
2) Booster parameters
3) Learning task parameters
4) Command line parameters

Before running a XGBoost model, we must set three types of parameters - general parameters, booster parameters and task parameters.

The fourth type of parameters are command line parameters. They are only used in the console version of XGBoost.

Some of the hyperparameters of XGBoost Algorithm is explained below :

1) eta/learning_rate: The shrinkage factor between 0 and 1 applied to α in the boosting algorithm. The default is 0.3, but for noisy data, smaller values are recommended (e.g., 0.1). 

2) n_rounds/n_estimators : The number of boosting rounds. If eta is set to a small value, it is important to increase the number of rounds since the algorithm learns more slowly.

3) max_depth : The maximum depth of the tree (the default is 6). In contrast to the random for‐ est, which fits very deep trees, boosting usually fits shallow trees. This has the advantage of avoiding spurious complex interactions in the model that can arise from noisy data.

4) subsample and colsample_bytree : Fraction of the records to sample without replacement and the fraction of predic‐ tors to sample for use in fitting the trees. These parameters, which are similar to those in random forests, help avoid overfitting.

5) lambda/reg_lambda and alpha/reg_alpha : The regularization parameters to help control overfitting .

Ridge and Lasso regression uses two different penalty functions. Ridge uses L2 where as lasso go with L1. In ridge regression, the penalty is the sum of the squares of the coefficients and for the Lasso, it’s the sum of the absolute values of the coefficients. It’s a shrinkage towards zero using an absolute value (L1 penalty) rather than a sum of squares(L2 penalty).

As we know that ridge regression can’t have zero coefficients. Here, we can either select all the coefficients or none of them whereas LASSO does both parameter shrinkage and variable selection automatically because it zero out the co-efficients of collinear variables. Here it helps to select the variable(s) out of given n variables while performing lasso regression.

Fitting a time series forecasting model requires 5 steps . The steps are explained below :

1) Data preparation : Data preparation is usually the first step where we load all the essential packages and data into a time series object.

2) Time series decomposition : Decomposition basically means deconstructing and visualizing the series into its component parts.

3) Modelling : The actual model building is a simple 2-lines code using auto.arima() function. auto.arima will take care of the optimum parameter values, we just need to specify a few boolean parameters.

4) Forecasting : Making an actual forecast is the simplest of all the steps above . We are using forecast() function and passing the model above and specifying the number of time steps into the future we want to forecast.

5) Model evaluation : This is an extra step for model evaluation and accuracy tests .

CNN : Convolutional layers . CNNs have unique layers called convolutional layers which separate them from RNNs and other neural networks. Within a convolutional layer, the input is transformed before being passed to the next layer. A CNN transforms the data by using filters.

RNN : Recurrent neural networks are networks that are designed to interpret temporal or sequential information. RNNs use other data points in a sequence to make better predictions. They do this by taking in input and reusing the activations of previous nodes or later nodes in the sequence to influence the output.

The main difference between CNN and RNN is the ability to process temporal information or data that comes in sequences, such as a sentence for example. Moreover, convolutional neural networks and recurrent neural networks are used for completely different purposes, and there are differences in the structures of the neural networks themselves to fit those different use cases.

CNNs employ filters within convolutional layers to transform data. Whereas, RNNs reuse activation functions from other data points in the sequence to generate the next output in a series.

Outlier values, or simply outliers, are data points in statistics that don’t belong to a certain population. An outlier value is an abnormal observation that is very much different from other values belonging to the set.

Identification of outlier values can be done by using univariate or some other graphical analysis method. Few outlier values can be assessed individually but assessing a large set of outlier values require the substitution of the same with either the 99th or the 1st percentile values.

There are two popular ways of treating outlier values:
1) To change the value so that it can be brought within a range
2) To simply remove the value

General Tip : Before an interview for any company , have a breif insight about the company , what it does , when was it founded and so on . All these info can be easily acquired from the Company Website itself .

Tip 1 : The cross questioning can go intense some time, think before you speak.
Tip 2 : Be open minded and answer whatever you are thinking, in these rounds I feel it is important to have opinion.
Tip 3 : Context of questions can be switched, pay attention to the details. It is okay to ask questions in these round, like what are the projects currently the company is investing, which team you are mentoring. How all is the work environment etc.
Tip 4 : Since everybody in the interview panel is from tech background, here too you can expect some technical questions. No coding in most of the cases but some discussions over the design can surely happen.